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Introduction 

The National Study on Alternate Assessments 
(NSAA) was mandated by Section 664(c) of 
the Individuals With Disabilities Education 
Improvement Act of 2004 (IDEA). Specifically, 
the law called for a “Study on Ensuring 
Accountability for Students Who Are Held 
to Alternative Achievement Standards” to 
examine the following: 

“(1) the criteria that States use to determine — 

(A) eligibility for alternate assessments; 
and 

(B) the number and type of children who 
take those assessments and are held 
accountable to alternative achievement 
standards; 

(2) the validity and reliability of alternate 
assessment instruments and procedures; 

(3) the alignment of alternate assessments 
and alternative achievement standards 
to State academic content standards in 
reading, mathematics, and science; and 

(4) the use and effectiveness of alternate 
assessments in appropriately measuring 
student progress and outcomes specific 
to individualized instructional need.” 

(P.L. 108-446, 118 Stat. 2784, 2004) 

SRI International and its partners Policy 
Studies Associates (PSA) and the National 
Center on Educational Outcomes (NCEO) 
were selected by the Institute of Education 
Sciences (IES) National Center for Special 
Education Research (NCSER) to conduct the 
NSAA. 



The alternate assessment arena is rapidly 
evolving across the 50 states and the 
District of Columbia, reflecting the states’ 
responses to legislative, regulatory, technical, 
and methodological developments in the 
assessment of students with significant 
cognitive disabilities for federal, state, and 
local accountability purposes. NSAA is 
examining alternate assessment systems 
during this period of rapid evolution using two 
primary data collection methods: document 
analysis and data verification activities in 
50 states and the District of Columbia, and 
teacher surveys in selected states. 

Two reports have been produced to document 
NSAA findings for the 2006-07 school year. 
The two reports present findings of the 
document analysis and data verification 
activities. State Profiles on Alternate 
Assessments Based on Alternate Achievement 
Standards. A Report From the National Study 
on Alternate Assessments (NSAA State 
Profiles) presents data summaries in the form 
of individual state profiles for the 2006-07 
school year for each of the 50 states and 
the District of Columbia. The NSAA State 
Profiles describe individual state approaches 
to designing and administering alternate 
assessments based on alternate achievement 
standards, key features of individual 
state alternate assessments, and student 
participation and performance data for each 
state. A companion report, the National Profile 
on Alternate Assessments Based on Alternate 
Achievement Standards. A Report From the 
National Study on Alternate Assessments 
(NSAA National Profile) summarizes national- 
level findings across the 50 states and the 
District of Columbia for the 2006-07 school 
year. The NSAA National Profile also contains 
data tables with individual state responses for 
each item of the data collection instrument. 
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Legislative Background 

As introduced in the IDEAAmendments of 
1997 (IDEA 1997), an alternate assessment 
is an assessment designed for children with 
disabilities who are unable to participate 
in the regular state assessment, even with 
appropriate accommodations (20 U.S.C. 
1412(a)(16)). IDEA 1997 required states to 
develop and conduct alternate assessments 
no later than July 1 , 2000. The statute did 
not place specific limits on the number of 
students who could participate in alternate 
assessments nor did it specify the content 
of alternate assessments. However, 
the discussion accompanying the final 
regulations to implement Part B of that law 
stated that “it should be necessary to use 
alternate assessments for a relatively small 
percentage of children with disabilities” and 
“alternate assessments need to be aligned 
with the general curriculum standards set 
for all students and should not be assumed 
appropriate only for those students with 
significant cognitive impairments” (64 Fed. 
Reg. 12564-12565 (Mar. 12, 1999); 34 
C.F.R. § 200). States responded to IDEA 
1997 by implementing a variety of alternate 
assessment approaches, including portfolios, 
checklists, and individualized education 
program (IEP) analysis, which varied in the 
degree of emphasis on general education 
curriculum content versus a separate 
curriculum to develop functional skills 
(Thompson and Thurlow 2000). 

Federal policies since IDEA 1997 have 
required increased integration of alternate 
assessments into general school systems for 
academic accountability. The No Child Left 
Behind Act of 2001 (NCLB) required states 
to adopt challenging academic standards 
that were “the same academic standards that 
the State applies to all schools and children 
in the State.” Two types of standards were 
required: academic content standards and 
academic achievement standards. The statute 



called for academic content standards that 
“(I) specify what children are expected to 
know and be able to do; (II) contain coherent 
and rigorous content; and (III) encourage the 
teaching of advanced skills” and academic 
achievement standards that “(I) are aligned 
with the State’s academic content standards; 

(II) describe two levels of high achievement 
(proficient and advanced) that determine how 
well children are mastering the material in 
the State academic content standards; and 

(III) describe a third level of achievement 
(basic) to provide complete information about 
the progress of the lower-achieving children 
toward mastering the proficient and advanced 
levels of achievement.” (20 U.S.C. 6311 § 
1111(b)(1)(D)). 

In 2002, regulations implementing the 
assessment provisions of NCLB stated that 
“the State’s academic assessment system 
must provide for one or more alternate 
assessments for a student with disabilities 
[who] cannot participate in all or part of the 
State assessments ... even with appropriate 
accommodations.” These regulations further 
required that “alternate assessments must 
yield results in at least reading/language arts, 
mathematics, and, beginning in the 2007-08 
school year, science” (67 Fed. Reg. 45041- 
45042 (Jul. 5, 2002); 34 C.F.R. § 200). 

On December 9, 2003, the U.S. Department 
of Education issued final regulations under 
NCLB permitting states to develop “through a 
documented and validated standards-setting 
process” alternate achievement standards for 
students with the most significant cognitive 
disabilities, “provided that those standards 
(1) Are aligned with the State’s academic 
content standards; (2) Promote access 
to the general curriculum; and (3) Reflect 
professional judgment of the highest 
achievement standards possible” (34 C.F.R. 

§ 200.1 (2003)). An alternate achievement 
standard is “an expectation of performance 
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that differs in complexity from a grade-level 
achievement standard” (68 Fed. Reg. 68699 
(Dec. 9, 2003)). States may include proficient 
and advanced scores of students with the 
most significant cognitive disabilities based 
on the alternate achievement standards in 
determining “adequate yearly progress,” 
provided that the number of those proficient or 
advanced scores at the state and local levels 
does not exceed 1 percent of all students in 
the grades assessed in reading/language 
arts and in mathematics, unless the state has 
received an exception permitting it to exceed 
this cap (34 C.F.R. § 200.13(c) (2003)). 

Description of NSAA State Profiles 

Data were collected in 2006 and 2007 through 
a process involving detailed analysis of state 
documents. In particular, submissions to the 
U.S. Department of Education’s Standards and 
Assessments Peer Review process served as 
the primary data source. Structured telephone 
interviews with knowledgeable informants 
in each of the 50 states and the District of 
Columbia were also conducted to gather 
information that could not be gleaned from the 
analysis of state documents. For simplicity, the 
District of Columbia is considered one of the 
51 “states.” The Standards and Assessment 
Peer Review process and data collection, 
verification, and analysis procedures are 
described in appendix A. The data collected 
were of four types: yes/no items, multiple- 
choice items, closed-ended text (such as 
the name of the assessment and number of 
content standards addressed), and open- 
ended responses. Open-ended responses 
were coded into categories. Descriptions of 
data collected and coding procedures are 
included in appendix B. This report does not 
reflect any updates that may have occurred 
since data collection activities ended in 
September 2007. A number of states had 
begun reworking their alternate assessments 
at that time, often in response to feedback 



from the Standards and Assessments Peer 
Review process, so their current and future 
alternate assessments may differ from the 
information presented in these reports. 

Each six-page state profile contains the 
following sections: 

A. Overview 

B. Alternate Achievement Standards 

C. Technical Quality 

D. Eligibility and Administration 

E. Scoring and Reporting 

F. Participation and Proficiency 

A brief description of each section is provided 
here. 

A: Overview. The Overview section 
presents selected key features of alternate 
assessments in 2006-07, including the 
purposes the state reported for the alternate 
assessment, the general approaches and 
procedures used, and the coverage of 
academic content standards. NCLB required 
states, beginning in 2005-06, to administer 
assessments in reading/language arts and 
in mathematics in each of grades 3 through 
8 and at least once in grades 10 through 12. 
Although states were required to develop 
achievement standards in science by 
2005-06, assessments in science were not 
required to be administered until 2007-08. 

B: Alternate Achievement Standards. The 

NCLB statute and regulations set forth certain 
requirements for academic achievement 
standards, and these requirements apply to 
alternate achievement standards, in addition 
to the requirements in the December 9, 

2003, regulation on alternate achievement 
standards. For example, alternate 
achievement standards must have the three 
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achievement levels required by NCLB, and 
“proficient” and “advanced” must represent 
high achievement while “basic” must represent 
achievement that is not yet proficient. These 
labels may vary from state to state, such as 
“mastering” or “exceeds standards” for the 
advanced labels, “independent” or “meets 
standards” for proficient, and “exploring” 
or “below the standard” for basic. A state 
may use more than three levels but must 
clearly indicate which level represents the 
proficiency expected of all students. The 
state’s academic achievement standards and 
alternate achievement standards must include 
descriptors of the competencies associated 
with each level. The state must also determine 
which specific scores on its assessments 
distinguish one level from another (34 C.F.R. § 
§ 200.1(c)). 

C: Technical Quality. NCLB requires that 
state assessments “be used for purposes 
for which such assessments are valid and 
reliable, and be consistent with relevant, 
nationally recognized professional and 
technical standards” (20 U.S.C. 6311 § 

1111 (b)(3)(C)(iii). The U.S. Department 
of Education’s (2004) Standards and 
Assessments Peer Review Guidance 
references the Standards for Educational 
and Psychological Testing (AERA/APA / 
NCME1999) to delineate the elements of 
validity and reliability required for technical 
quality. The elements of validity and reliability 
of assessment items (e.g., scoring and 
reporting structures, test and item scores, 
purposes of the assessment, grade-level 
equating) were based on the elements 
included in the Standards and Assessments 
Peer Review Guidance. 

Other dimensions of technical quality reported 
here include fairness/accessibility, consistent 
procedures for test implementation, and 
alignment. Also reported here is the use 
of “extended” content standards. Alternate 
assessments based on alternate achievement 



standards must be aligned with the content 
standards for the grade in which the student 
is enrolled, although the grade-level content 
may be reduced in complexity or modified to 
reflect prerequisite skills. States can adapt or 
“extend” these grade-level content standards 
to reflect instructional activities appropriate for 
students with the most significant cognitive 
disabilities (U.S. Department of Education 
2005). 

D: Eligibility and Administration. The 

regulations for alternate achievement 
standards require states to establish 
guidelines for individualized education 
program (IEP) teams to use in identifying 
children with the “most significant cognitive 
disabilities” who will be assessed based 
on alternate achievement standards. The 
regulations do not prescribe a federal 
definition of “the most significant cognitive 
disabilities,” nor do they set federal guidelines. 
The regulations require that the state ensure 
that parents be informed that their child’s 
achievement will be based on alternate 
achievement standards (34 C.F.R. § 200.1(f)). 

States have considerable flexibility in 
designing their alternate assessments, 
provided the statutory and regulatory 
requirements are met. The general alternate 
assessment approaches the states used 
in 2006-07 were reported in the Overview 
section, but states sometimes used these 
approaches in combination, and each 
approach could be implemented in varying 
ways. 

E: Scoring and Reporting. NCLB requires 
states to produce “interpretive, descriptive, 
and diagnostic reports” on individual students’ 
achievement measured against academic 
achievement standards to help parents, 
teachers, and principals to address the 
academic needs of students (20 U.S.C. 6311 § 
1111 (b)(3)(C)(xii); 34 C.F.R. § 200.8). Scoring 
criteria for students with the most significant 
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cognitive disability may include elements 
typically found in general assessments, 
such as accuracy, and elements selected 
specifically for this population, such as 
independence, progress, and generalization 
across multiple settings. 

F: Participation and Proficiency. The 

Participation and Proficiency pages of the 
state profiles display state-reported data for 
the 2006-07 academic year, as discussed 
below. 

A template is provided on page 7 to assist 
the reader in understanding the source 
of the information in the state profiles. 

The template indicates, with variable 
identifiers in bold, specific items used for 
data collection. Further explanation of each 
item is contained in appendix B, with items 
arranged according to variable identifiers. 
Following the template are 52 state profiles, 
including the District of Columbia (Michigan 
has two profiles corresponding to the two 
alternate assessments that state used in 
2006-07). Each profile item lists the possible 
response categories, and check marks 
indicate the category or categories that apply 
to the specific state. For example, if a state 
reported that an assessment company and 
state personnel were involved in developing 
its alternate assessment, then check marks 
appear next to those two items in the list of 
possible responses. 

In cases where data omissions exist (e.g., 
because information was not reported by a 
state or special circumstances pertained to 
a state), explanatory footnotes are included. 
For the technical quality items, if no check 
mark is included in the “type of validity,” “type 
of reliability,” or “type of technical quality” 
column, then the state either did not document 
that particular type of technical quality of the 
alternate assessment or the state reported that 
the type of evidence of technical quality was 
not appropriate for its particular assessment 
approach. 



The last page of each state profile displays the 
participation and proficiency data of students 
with disabilities who participate in alternate 
assessments based on alternate achievement 
standards and other assessments and the 
performance of all students including students 
with disabilities. The charts titled “Ways 
that students with lEPs participated in state 
accountability assessments” indicate the 
percentage of the population of students with 
lEPs (i.e., identified as having a disability 
under IDEA) that participated in alternate 
assessments based on alternate achievement 
standards, versus those who participated 
in other forms of assessment (e.g., regular 
assessments) or were absent or exempt. 
Students could be exempt for a number of 
reasons (e.g., a student moved out of the 
state before testing began, or a student was 
counted and then suspended/expelled/jailed 
during the testing window). In providing these 
participation data, states did not report the 
number of cases in which assessment results 
were considered invalid, for example, because 
of nonstandard accommodations. Thus, an 
unknown portion of the percentage reported as 
participating did not produce scores that were 
used for accountability purposes. 

The graphs titled “Percentage of students 
who scored proficient or above on state 
accountability assessments” indicate the 
percentage of students with lEPs, or of the 
general student population, who scored 
proficient or above on the specified type of 
assessment, out of all the students who took 
that type of assessment and received a valid 
score. 

The table titled “Percentage of students in 
the grades assessed who scored proficient or 
above on an alternate assessment based on 
alternate achievement standards” indicates 
percentages based on a denominator of all the 
students in the grades assessed. As such, it 
parallels the language in 34 C.F.R. § 200.13(c) 
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that places the 1 percent cap on the number 
of students who can be counted as proficient 
or advanced on alternate assessments 
based on alternate achievement standards 
in determining adequate yearly progress. 
However, the table should not be interpreted 
as an authoritative indication that the state did 
or did not exceed this cap. The table presents 
overall state percentage by grade level and 
content area, not those for local education 
agencies to which the cap also applies, 
and it may not have taken into account all 
the considerations used in calculating an 
authoritative percentage. 

The primary source of data for the NSAA 
profiles was an in depth review of documents 
submitted to the U.S. Department of Education 
in response to the Standards and Assessment 
Peer Review process and information 
pertaining to the alternate assessments 
on state websites. The study team also 
conducted structured telephone interviews 
with knowledgeable informants in each of 
the 50 states and the District of Columbia. 
These interviews were not systematic surveys. 
Rather, they were conducted to verify the 
results of document review and to collect data 
that could not be collected from administrative 
records. The study methodology is presented 
in appendix A and collected data were 
coded according to procedures described in 
appendix B. 



Special Notes 

There are a few special notes to be aware of 

for the NSAA State Profiles'. 

• Michigan reported having two alternate 
assessments based on alternate 
achievement standards in use in 2006-07. 
In this report, each alternate assessment 
for Michigan is presented in a separate 
profile. 

• Florida did not have an alternate 
assessment based on alternate 
achievement standards in 2006-07 and 
was revising its alternate assessment 
system at that time. The report includes 
data for Florida with the caveat to use 
caution in interpreting data for that state. 

• There are currently several allowable 
variants of alternate assessments, 
including alternate assessments based 
on alternate achievement standards, 
alternate assessments based on 
modified achievement standards, 

and alternate assessments based on 
grade-level standards. Only alternate 
assessments based on alternate 
achievement standards are included 
in this NSAA report. Throughout the 
report, alternate assessments based 
on alternate achievement standards 
may be referred to simply as “alternate 
assessments” for brevity. In such 
cases it should be understood that 
only alternate assessments based on 
alternate achievement standards are being 
considered. 
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Overview of the Alabama Alternate Assessment (AAA) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Alabama Alternate 
Assessment (AAA) were to 

Y Evaluate programs 

Y Guide classroom instruction 

Y Measure student progress/performance toward state standards 

Y Assess student access to content standards/general curriculum 

Y Assess individual students’ strengths/weaknesses 

Y Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

Alabama Alternate Assessment (AAA) 
Alternate Assessment Approaches 
Rating scale/checklist 

Y Portfolio/body of evidence 
Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 
Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
Varied by grade level 
Mathematics 

Varied by grade level 

Who was involved in developing the alternate 
assessment? 

Y Assessment company 

Y Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 
Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Student’s classroom teacher 
School- or district-based educator 

Y State or state-contracted scorer 
Machine scored 
Paraprofessional 



V Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

Alabama Alternate Assessment for the 2006-07 School Year 
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Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 
•A State instruction and curriculum staff 
Test vendor 
•A Outside experts 
•A Special education teachers 
•A General education teachers 
•A Content specialists 

School psychologists/counselors 
•A School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Performance Profile Method and 
the Contrasting Groups standard-setting 
methodologies. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Alabama 
proficiency levels for 
alternate assessment 


Advanced 


Exceeds Standards: Level IV 


Proficient 


Meets Standards: Level III 


Basic 


Partially Meets Standards: Level II 




Does Not Meet Standards: 




Level 1 



Students who performed at the Exceeds Standards 
and Meets Standards levels were proficient for 
purposes of the No Child Left Behind Act, whereas 
students who scored at the Partially Meets 
Standards and Does Not Meet Standards levels 
were not proficient. 



Alternate Achievement Descriptors 

Alabama developed 

The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

•A Descriptors that were unique for each grade 
tested 

An example of Level III Meets Standards descriptor 

in reading/language arts is as follows: 

• The Level III student has demonstrated 
progress on the extended standards 
in reading at the level of complexity 
identified by the IEP Team. The complexity 
is determined by the student’s level of 
functioning and educational expectations as 
defined in the IEP. This student either made 
progress toward more complex skills with 
some prompting or less complex skills with 
independence. Grade 5 Extended Standards 
Reading: identify words with more than one 
meaning; draw conclusions about characters 
based on their actions; use a range of 
strategies and skills including phonetic skills 
to read unfamiliar and/or multi-syllable words. 

Alternate Achievement Cut Scores 

Alabama developed 

•A Unique cut scores for each grade 
Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



V Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

National Study on Alternate Assessments: State Profiles 
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Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 








✓ 


State assessment staff 


✓ 


✓ 




✓ 


State instruction and curriculum staff 










Test vendor 


✓ 




✓ 


✓ 


Outside experts 


■/ 






✓ 



Special education teachers 
General education teachers 
Content specialists 
School psychologists/counselors 
School/district/state administrators 
Parents 



Validity 



Reliability 



Type of validity Types of evidence 





Scoring and reporting 
structures consistent with 
the subdomain 






Relation to internal and 
external variables 




✓ 


Purposes of the assessment 


Not available for 
examination 


✓ 


Intended and unintended 
consequences 


Not available for 
examination 


✓ 


Measurement of construct 
relevance 


Not available for 
examination 




Grade-level equating 





Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 

Y Grade or grade span 

Y Expanded benchmarks 
Alternate indicators or tasks 



Type of reliability 


Types of evidence 


✓ 


Variability across groups 


Not available for 
examination 


✓ 


Internal consistency of item 
responses 


Not available for 
examination 


✓ 


Interrater consistency in 
scoring 


Not available for 
examination 


✓ 


Standard errors of 
measurement 


Statistical analysis 


Other Types of Technical Quality 


Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Anecdotal or 
committee process 


✓ 


Ongoing/monitoring 


Other alignment 
studies 




Fairness 




✓ 


Implementation processes 


Training 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

Alabama Alternate Assessment for the 2006-07 School Year 
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Eligibility and Administration 



Eligibility 

Alabama provided the following guidelines to be 
used in determining when a child’s significant 
cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. 

Assessment Content Selection 

State determined 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

•A Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
•A Student work samples only 

Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 
•A As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

•A Teacher decided content to be assessed 
•A Teacher selected materials 

Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

National Study on Alternate Assessments: State Profiles 



10 




Alabama 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

One scorer 

• Who scored? 

State or state-contracted scorer 

• How were scoring conflicts resolved? 

A third person adjudicated 
Y A third rater scored the assessment 

One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 
Ability to generalize across settings 

Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

Y None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 






Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 


✓ 


✓ 



Individual Student Reports 

Individual student results were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
Y' Scores 

Percentiles 

The state indicated there was no interpretive 
guidance in place. 

Reports given to parents included the following 
information 

Y Performance/achievement level 
■S Scores 

Y Standard/strand breakouts 
Indicator/benchmark breakouts 
Performance/achievement level descriptors 
Sample test items 



f \ 

State department of education 
website 

www.alsde.edu 

V ) 



V Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

Alabama Alternate Assessment for the 2006-07 School Year 
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Alabama 



Participation and Proficiency (2006-07) 

Ways that students with lEPs participated in state accountability assessments 

Reading/Language Arts Mathematics 




Percent 

□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 




Percent 

□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts 




Percent 



Mathematics 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ Students with an IEP taking a regular assessment 
with or without accommodations 

□All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ Students with an IEP taking a regular assessment 
with or without accommodations 

□All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.48 


0.48 


6-8 


0.51 


0.54 


9-12 


0.56 


0.53 



M Missing. Information not provided by state. 

1 0f all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, EDFacts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 




Alaska 



Overview of the Extended Reading, Writing, Math, and Science Alternate 
Assessment (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Extended Reading, 
Writing, Math, and Science Alternate Assessment were to 

Y Evaluate programs 

Y Guide classroom instruction 

Y Measure student progress/performance toward state standards 

Y Assess student access to content standards/general curriculum 

Y Assess individual students’ strengths/weaknesses 

Y Document academic achievement 

Y Measure student progress toward IEP goals 



Alternate Assessment 

Extended Reading, Writing, Math, and Science 
Alternate Assessment 

Alternate Assessment Approaches 

Rating scale/checklist 
Portfolio/body of evidence 

Y Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 
Science 
Social studies 
Functional skills 

Grades Assessed 

ssssvsvv 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

Y More than 1 month to 2 months 
More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
Varied by grade level 
Mathematics 

Varied by grade level 

Who was involved in developing the alternate 
assessment? 

Y Assessment company 

Y Research company/university/independent 
researcher 

Y Technical assistance provider 

Y State personnel 

Y Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Y Student’s classroom teacher 
School- or district-based educator 
State or state-contracted scorer 
Machine scored 
Paraprofessional 



V Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

Alaska Alternate Assessment for the 2006-07 School Year 
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Alaska 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 

•A State assessment staff 

•A State instruction and curriculum staff 

s Test vendor 

•A Outside experts 

•A Special education teachers 

•A General education teachers 

•A Content specialists 

•A School psychologists/counselors 

A School/district/state administrators 

A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Reasoned Judgment standard-setting 
methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Alaska 
proficiency levels for 
alternate assessment 


Advanced 


Advanced 


Proficient 


Proficient 


Basic 


Below Proficient 




Far Below Proficient 



Alternate Achievement Descriptors 

Alaska developed 

The same descriptors that applied to all 
grades tested 

A The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Proficient descriptor in reading/ 
language arts (grades 3 and 4) is as follows: 

• The student reproduces/copies words 
using upper and lower case letters; writes/ 
reproduces own first name; orients graphics 
in legible format (right-side up, left to right). 

Alternate Achievement Cut Scores 

Alaska developed 

Unique cut scores for each grade 
A Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Advanced and 
Proficient levels were proficient for purposes of 
the No Child Left Behind Act, whereas students 
who scored at the Below Proficient and Far Below 
Proficient levels were not proficient. 



A Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

National Study on Alternate Assessments: State Profiles 
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Alaska 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 








✓ 


State assessment staff 


✓ 


✓ 




✓ 


State instruction and curriculum staff 








✓ 


Test vendor 


✓ 


✓ 


✓ 


✓ 


Outside experts 


✓ 


✓ 


✓ 


✓ 


Special education teachers 






✓ 


✓ 


General education teachers 






✓ 


✓ 


Content specialists 






✓ 


✓ 


School psychologists/counselors 










School/district/state administrators 










Parents 






✓ 


✓ 



Validity 



Reliability 



Type of validity Types of evidence 

Scoring and reporting 
structures consistent with 
the subdomain 



✓ 


Relation to internal and 
external variables 


Not available for 
examination 


✓ 


Purposes of the assessment 


Not available for 
examination 


✓ 


Intended and unintended 
consequences 


Survey 


✓ 


Measurement of construct 
relevance 


Not available for 
examination 


✓ 


Grade-level equating 


Not available for 
examination 



Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 

Y Grade or grade span 
•Y Expanded benchmarks 

Alternate indicators or tasks 



Type of reliability 


Types of evidence 


✓ 


Variability across groups 


NCLB group statistical 
analyses 


✓ 


Internal consistency of item 


Not available for 




responses 

Interrater consistency in 
scoring 


examination 


✓ 


Standard errors of 
measurement 


Statistical analysis 



Other Types of Technical Quality 



Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Formal study 


✓ 


Ongoing/monitoring 


External alignment 
study 


✓ 


Fairness 


Bias review, not 
conducted regularly 


✓ 


Implementation processes 


Not available for 
examination 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

Alaska Alternate Assessment for the 2006-07 School Year 
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Alaska 



Eligibility and Administration 



Eligibility 

Alaska provided the following guidelines to be used 
in determining when a child’s significant cognitive 
disability justified an alternate assessment based 
on alternate achievement standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

■A Required extensive support for skill 
generalization 

Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

•A Parent signature was required 

Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

•A All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

•A State required standardized tasks/test items/ 
rating scales 

State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

Combination of work samples and other 
evidence 

•A No student work samples 

Assessment was administered 

As part of day-to-day student instruction 
•A Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

Teacher decided content to be assessed 
Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

National Study on Alternate Assessments: State Profiles 
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Alaska 



Scoring and Reporting 
Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

One scorer 

• Who scored? 

Student’s classroom teacher 

• How were scoring conflicts resolved? 

A third person adjudicated 
A third rater scored the assessment 

Y' One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 
Ability to generalize across settings 

Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

Y None of the above 



Training 





Assessment 


Assessment 




administrators 


scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


Y 


✓ 


Mandatory 
training and/or 
certification 


Y 


✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Schools and teachers 

Individual student results were expressed 
in terms of 

S State’s achievement standards 
Scores 

Y Percentiles 

Interpretive guidance was developed for 
S School-level administrators 

Y Teachers 
S Parents 

Students 

Reports given to parents included the following 
information 

S Performance/achievement level 
Y' Scores 

Y Standard/strand breakouts 
Indicator/benchmark breakouts 

Y Performance/achievement level descriptors 

Y Sample test items 



f \ 

State department of education 
website 

www.eed.state.ak.us 

V ) 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

Alaska Alternate Assessment for the 2006-07 School Year 
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Alaska 



Participation and Proficiency (2006-07) 

Ways that students with lEPs participated in state accountability assessments 

Reading/Language Arts Mathematics 




Percent 

□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 




Percent 

□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts 




Percent 



Mathematics 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.28 


0.43 


6-8 


0.25 


0.32 


9-12 


0.33 


0.36 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, EDFacfs - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 



Arizona 



Overview of Arizona’s Instrument to Measure Standards-Alternate (AIMS-A) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Arizona’s Instrument to 
Measure Standards-Alternate (AIMS-A) were to 

Y Evaluate programs 

•A Guide classroom instruction 

Y Measure student progress/performance toward state standards 
Assess student access to content standards/general curriculum 

Y Assess individual students’ strengths/weaknesses 

Y Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

Arizona’s Instrument to Measure Standards- 
Alternate (AIMS-A) 

Alternate Assessment Approaches 

Y Rating scale/checklist 
Portfolio/body of evidence 

Y Performance task/events 

Y Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 
Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
3 of 3 standards 
Mathematics 

5 of 5 standards 

Who was involved in developing the alternate 
assessment? 

Assessment company 

Y Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 

Y Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Y Student’s classroom teacher 
School- or district-based educator 
State or state-contracted scorer 
Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

Arizona Alternate Assessment for the 2006-07 School Year 
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Arizona 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
State assessment staff 
State instruction and curriculum staff 
Test vendor 
Outside experts 
•A Special education teachers 
•A General education teachers 
•A Content specialists 
•A School psychologists/counselors 
School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Bookmark or Item Mapping standard- 
setting methodology. 



Alternate Achievement Descriptors 

Arizona developed 

The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

•A Descriptors that were unique for each grade 
tested 

An example of the Meets the Standard descriptor in 
reading/language arts is as follows: 

• Students who score in this level and are 
supported through the use of symbols, 
pictures and objects, demonstrate an 
understanding of subject matter as reflected 
by the reading standard. Students who 
perform at this level are able to determine 
the meaning of simple words, retell a story in 
sequence and identify specific facts from a 
text. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Arizona 
proficiency levels for 
alternate assessment 


Advanced 


Exceeds the Standards 


Proficient 


Meets the Standards 


Basic 


Approaches the Standards 




Falls Far Below the Standards 



Alternate Achievement Cut Scores 

Arizona developed 
•A Unique cut scores for each grade 
Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Exceeds the 
Standards and Meets the Standards levels 
were proficient for purposes of the No Child Left 
Behind Act, whereas students who scored at the 
Approaches the Standards and Falls Far Below the 
Standards levels were not proficient. 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

National Study on Alternate Assessments: State Profiles 
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Arizona 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


Z 


Z 


Z 


Z 


State assessment staff 


z 


Z 


Z 


Z 


State instruction and curriculum staff 










Test vendor 










Outside experts 


z 


Z 


z 


Z 


Special education teachers 


z 


Z 


z 


Z 


General education teachers 


z 




z 




Content specialists 


z 




z 




School psychologists/counselors 


z 




z 


Z 


School/d istri ct/state ad m i n i strata rs 


z 




z 


Z 


Parents 











Validity 



Type of validity 


Types of evidence 


Z 


Scoring and reporting 
structures consistent with 
the subdomain 


Scoring and reporting 
documents 


Z 


Relation to internal and 
external variables 


Correlational analysis 
using external 
measures 


Z 


Purposes of the assessment 


Alignment study 


Z 


Intended and unintended 
consequences 


Survey 


Z 


Measurement of construct 
relevance 


Statistical analyses 




Grade-level equating 





Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 

Y Grade or grade span 

Y Expanded benchmarks 
Alternate indicators or tasks 



Reliability 



Type of reliability 


Types of evidence 


✓ 


Variability across groups 


NCLB group 
statistical analyses 


✓ 


Internal consistency of item 
responses 

Interrater consistency in 
scoring 


Formal study or expert 
panel review 


✓ 


Standard errors of 
measurement 


Statistical analysis 
conducted as part of 
actual scoring 


Other Types of Technical Quality 


Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Formal study 


✓ 


Ongoing/monitoring 


External alignment 
study 


✓ 


Fairness 


Bias review, not 
conducted regularly 


✓ 


Implementation processes 


Training, monitoring 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

Arizona Alternate Assessment for the 2006-07 School Year 
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Arizona 



Eligibility and Administration 



Eligibility 

Arizona provided the following guidelines to be 
used in determining when a child’s significant 
cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

Required extensive support for skill 
generalization 

•A Required modified curriculum 
Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. 

Assessment Content Selection 

State determined 

•A All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

•A No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

•A State required standardized tasks/test items/ 
rating scales 

State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

Combination of work samples and other 
evidence 

•A No student work samples 

Assessment was administered 

As part of day-to-day student instruction 
Separately from student’s daily work 

•A Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

Teacher decided content to be assessed 
Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

National Study on Alternate Assessments: State Profiles 
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Arizona 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

One scorer 

• Who scored? 

Student’s classroom teacher 

• How were scoring conflicts resolved? 

A third person adjudicated 
A third rater scored the assessment 

Y One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 

Y Ability to generalize across settings 

Y Amount of independence 

Y Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

Y None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 







Individual Student Reports 

Individual student results were given to 

Y Parents 

Y Schools and teachers 

Individual student reports were expressed 
in terms of 

Y State’s achievement standards 
Scores 

Y Percentiles 

Interpretive guidance was developed for 
School-level administrators 

Y Teachers 

Y Parents 

Y Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
■S Scores 

Y Standard/strand breakouts 
Indicator/benchmark breakouts 

Y Performance/achievement level descriptors 
Sample test items 



f \ 

State department of education 
website 

www.ade.az.gov 

V ) 



■/ Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

Arizona Alternate Assessment for the 2006-07 School Year 
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Arizona 



Participation and Proficiency (2006-07) 

Ways that students with lEPs participated in state accountability assessments 

Reading/Language Arts Mathematics 



Grades 




Percent 

□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 




Percent 

□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Grades 



3-5 



6-8 



9-12 



Reading/Language Arts 



Grades 



166 


165 

[38 


m 3-5 


^]68 


172 






1 66 


165 


123 6-8 


122 


^]66 


^|65 






164 


^|66 


126 9-12 




|72 


167 



Mathematics 



0 20 40 60 80 100 

Percent 



— i — 
0 20 



40 60 80 

Percent 



100 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.51 


0.50 


6-8 


0.59 


0.57 


9-12 


0.56 


0.58 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 



Arkansas 



Overview of the Arkansas Alternate Portfolio Assessment System for Students 
with Disabilities (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Arkansas Alternate 
Portfolio Assessment System for Students with Disabilities were to 

Y Evaluate programs 

Y Guide classroom instruction 

Y Measure student progress/performance toward state standards 

Y Assess student access to content standards/general curriculum 
Assess individual students’ strengths/weaknesses 
Document academic achievement 

Measure student progress toward IEP goals 



General Information 
Alternate Assessment 

Arkansas Alternate Portfolio Assessment System 
for Students with Disabilities 

Alternate Assessment Approaches 

Rating scale/checklist 

Y Portfolio/body of evidence 
Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 

Y Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
3 of 4 standards 
Mathematics 

5 of 5 standards 

Who was involved in developing the alternate 
assessment? 

Assessment company 

Research company/university/independent 
researcher 

Y Technical assistance provider 

Y State personnel 
Parents 
Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Student’s classroom teacher 
School- or district-based educator 

Y State or state-contracted scorer 
Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

Arkansas Alternate Assessment for the 2006-07 School Year 
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Arkansas 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 
•A State instruction and curriculum staff 
•A Test vendor 
Outside experts 
•A Special education teachers 
•A General education teachers 
■A Content specialists 

School psychologists/counselors 
School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Body of Work standard-setting 
methodology. 



Alternate Achievement Descriptors 

Arkansas developed 

•A The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Functional Independence 
descriptor in reading/language arts is as follows: 

• Students at the Functional Independence 
Level frequently meet authentic, age 
appropriate challenges. They demonstrate 
reasonable performance in multiple settings 
and are prepared for more challenging 
tasks. They can apply established literacy or 
mathematics skills to real-world situations but 
may require minimal prompting or support. 
They perform these skills accurately in most 
instances but make occasional errors. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Arkansas 
proficiency levels for 
alternate assessment 


Advanced 


Independent 


Proficient 


Functional Independence 


Basic 


Supported Independence 




Emergent 



Alternate Achievement Cut Scores 

Arkansas developed 
•A Unique cut scores for each grade 
Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Independent and 
Functional Independence levels were proficient 
for purposes of the No Child Left Behind Act, 
whereas students who scored at the Supported 
Independence and Emergent levels were not 
proficient. 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

National Study on Alternate Assessments: State Profiles 
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Arkansas 



Technical Quality 



Who was involved in reviewing the technical quality of the alternate assessment 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


✓ 


✓ 


✓ 




State assessment staff 


✓ 


✓ 


✓ 


✓ 


State instruction and curriculum staff 


✓ 


✓ 


✓ 


✓ 


Test vendor 


✓ 


✓ 


✓ 




Outside experts 




✓ 


✓ 




Special education teachers 


✓ 


✓ 


✓ 


✓ 


General education teachers 


✓ 


✓ 


✓ 


✓ 


Content specialists 


✓ 


✓ 


✓ 


✓ 



School psychologists/counselors 
School/district/state administrators 

Parents ■/ 



Validity 



Reliability 



Type of validity Types of evidence 



Scoring and reporting 
structures consistent with 
the subdomain 




Relation to internal and 
external variables 




S Purposes of the assessment 


Not available for 
examination 


Intended and unintended 
consequences 




✓ Measurement of construct 
relevance 


Construct analyses 


Grade-level equating 





Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

•Y General link to state content standards 

Y Grade or grade span 

Y Expanded benchmarks 

Y Alternate indicators or tasks 



Type of reliability Types of evidence 



Variability across groups 

Internal consistency of item 
responses 

V Interrater consistency in 
scoring 

Standard errors of 
measurement 


Training documents or 
anecdotal 


Other Types of Technical Quality 


Type of technical quality 


Types of evidence 


Alignment: 


■/ Initial 


Formal study 


■/ Ongoing/monitoring 


Internal alignment 
studies 


Fairness 


■/ Implementation processes 


Training, 
administration manual/ 
guide 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

Arkansas Alternate Assessment for the 2006-07 School Year 
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Arkansas 



Eligibility and Administration 



Eligibility 

Arkansas provided the following guidelines to 
be used in determining when a child’s significant 
cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

•A Parent signature was required 

Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

•A Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
•A Student work samples only 

Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 
•A As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

•A Teacher decided content to be assessed 
•A Teacher selected materials 

Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

National Study on Alternate Assessments: State Profiles 
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Arkansas 



Scoring and Reporting 
Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

Two scorers 

• Who scored? 

State or state-contracted scorer 

• How were scoring conflicts resolved? 

Y A third person adjudicated 

A third rater scored the assessment 

One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 

•Y Ability to generalize across settings 

Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 

Y Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 




Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 




✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 

Y Scores 
Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 

Y Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 

Y Scores 

Y Standard/strand breakouts 

Y Indicator/benchmark breakouts 

Y Performance/achievement level descriptors 
Sample test items 



f \ 

State department of education 

website 

www.arkansased.org 

V ) 



V Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

Arkansas Alternate Assessment for the 2006-07 School Year 
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Arkansas 



Participation and Proficiency (2006-07) 

Ways that students with lEPs participated in state accountability assessments 

Reading/Language Arts Mathematics 



Grades 




Percent 

□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Grades 



if] 


88 


[1 






2 


EFT 


88 




M 

1 


1 1 


1 1 



0 20 40 60 80 100 

Percent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts 




Percent 



Mathematics 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ Students with an IEP taking a regular assessment 
with or without accommodations 

□All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ Students with an IEP taking a regular assessment 
with or without accommodations 

□All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.89 


0.87 


6-8 


0.79 


0.69 


9-12 


1.64 


M 



M Missing. Information not provided by state. 

1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 



California 



Overview of the California Alternate Performance Assessment (CAPA) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 California Alternate 
Performance Assessment (CAPA) were to 

Evaluate programs 
•Y Guide classroom instruction 

•Y Measure student progress/performance toward state standards 

Y Assess student access to content standards/general curriculum 
Assess individual students’ strengths/weaknesses 

Y Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

California Alternate Performance Assessment 
(CAPA) 

Alternate Assessment Approaches 

Rating scale/checklist 
Portfolio/body of evidence 

Y Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 

Y Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓✓✓✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

Y More than 2 weeks to 1 month 
More than 1 month to 2 months 
More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
Varied by grade level 
Mathematics 

Varied by grade level 

Who was involved in developing the alternate 
assessment? 

Y Assessment company 

Y Research company/university/independent 
researcher 

Y Technical assistance provider 

Y State personnel 
Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

Y A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Y Student’s classroom teacher 

Y School- or district-based educator 
State or state-contracted scorer 
Machine scored 
Paraprofessional 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

California Alternate Assessment for the 2006-07 School Year 
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California 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 

•A State assessment staff 

•A State instruction and curriculum staff 

•A Test vendor 

•A Outside experts 

•A Special education teachers 

•A General education teachers 

•A Content specialists 

School psychologists/counselors 
•A School/district/state administrators 
Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Bookmark or Item Mapping standard- 
setting methodology. 



Alternate Achievement Descriptors 

California developed 

•A The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Proficient descriptor in reading/ 
language arts is as follows: 

• Level II students performing at the Proficient 
level show interest in print or symbols. They 
can recognize their names, some letters, and 
one or two printed words or symbols. They 
can follow one-step directions of two words 
or more using symbols or words. They can 
write or produce a partial representation of 
their name. When asked to choose between 
two options, they can indicate a preference. 
They can communicate using single words 
and an occasional phrase. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding California 
proficiency levels for 
alternate assessment 


Advanced 


Advanced 


Proficient 


Proficient 


Basic 


Basic 




Below Basic 




Far Below Basic 



Alternate Achievement Cut Scores 

California developed 

Unique cut scores for each grade 
Unique cut scores for grade spans 
•A One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Advanced and 
Proficient levels were proficient for purposes of the 
No Child Left Behind Act, whereas students who 
scored at the Basic, Below Basic, and Far Below 
Basic levels were not proficient. 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

National Study on Alternate Assessments: State Profiles 
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California 



Technical Quality 



Who was involved in reviewing the technical quality of the alternate assessment? 





Stakeholders Validity 


Reliability 


Alignment 


Fairness 




State special education staff 


✓ 




✓ 


✓ 




State assessment staff 


✓ 


✓ 


✓ 


✓ 




State instruction and curriculum staff 






✓ 






Test vendor 


✓ 


✓ 


✓ 


✓ 




Outside experts 


✓ 


✓ 


✓ 


✓ 




Special education teachers 


✓ 


✓ 


✓ 


✓ 




General education teachers 


✓ 


✓ 


✓ 


✓ 




Content specialists 


✓ 




✓ 


✓ 




School psychologists/counselors 












School/district/state administrators 


✓ 


✓ 


✓ 


✓ 




Parents 










Validity 


Reliability 






Type of validity Types of evidence 


Type of reliability 




Types of evidence 




Scoring and reporting 


✓ 


Variability across groups 


NCLB group statistical 




structures consistent with 








analyses, review of 




the subdomain 








disability group results 


✓ 


Relation to internal and Correlational study 


✓ 


Internal consistency of item 


Formal study or expert 




external variables 




responses 




panel review 




Purposes of the assessment 


✓ 


Interrater consistency in 


Statistical analysis 


✓ 


Intended and unintended 
consequences 

Measurement of construct Construct analyses 


✓ 


scoring 

Standard errors of 
measurement 


conducted as part of 
actual scoring 

Statistical analysis 



•/ Grade-level equating 



Anecdotal or 
committee process 



Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

•Y General link to state content standards 
Grade or grade span 
Expanded benchmarks 
Alternate indicators or tasks 



Other Types of Technical Quality 


Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Formal study 


✓ 


Ongoing/monitoring 


External alignment 
study 


✓ 


Fairness 


Regularly scheduled 
bias review by experts, 
statistical analyses 


✓ 


Implementation processes 


Training, 
administration manual/ 
guide, monitoring 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

California Alternate Assessment for the 2006-07 School Year 
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California 



Eligibility and Administration 



Eligibility 

California provided the following guidelines to 
be used in determining when a child’s significant 
cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

•A Parent signature was required 

•A Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

•A All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

•A A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

•A State required standardized tasks/test items/ 
rating scales 

State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

Combination of work samples and other 
evidence 

•A No student work samples 

Assessment was administered 

As part of day-to-day student instruction 
•A Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

Teacher decided content to be assessed 
Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

National Study on Alternate Assessments: State Profiles 
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California 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

One scorer 

• Who scored? 

Student’s classroom teacher, school- or 
district-based educator 

• How were scoring conflicts resolved? 

A third person adjudicated 
A third rater scored the assessment 

Y One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 
Ability to generalize across settings 

•Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

Y None of the above 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

State’s achievement standards 

Y Scores 

Y Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 

Y Scores 

Standard/strand breakouts 
Indicator/benchmark breakouts 

Y Performance/achievement level descriptors 
Sample test items 



Training 





Assessment 


Assessment 


r 


\ 


Non-face-to-face 

training 


administrators 

✓ 


scorers 


State department of education 
website 




Face-to-face 

training 


✓ 


✓ 


www. cd e . ca . go v/ta/tg/s r 

V 


) 



Mandatory 


✓ 


✓ 


training and/or 






certification 







S Check marks indicate categories reported by the state from among those possible. 



SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

California Alternate Assessment for the 2006-07 School Year 
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California 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Mathematics 




Grades 




Percent 



Percent 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts 




Percent 



Mathematics 




0 20 40 60 80 100 



Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.60 


0.60 


6-8 


0.56 


0.43 


9-12 


0.54 


0.44 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 




Colorado 



Overview of the Colorado Student Assessment Program Alternate (CSAPA) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Colorado Student 
Assessment Program Alternate (CSAPA) were to 

Evaluate programs 
Guide classroom instruction 

Y Measure student progress/performance toward state standards 

Y Assess student access to content standards/general curriculum 

Y Assess individual students’ strengths/weaknesses 
Document academic achievement 

Measure student progress toward IEP goals 



Alternate Assessment 

Colorado Student Assessment Program Alternate 
(CSAPA) 

Alternate Assessment Approaches 

Rating scale/checklist 
Portfolio/body of evidence 

Y Performance task/events 

Y Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 

Y Science 
Social studies 
Functional skills 

Grades Assessed 

YYYYYV'YY 
3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

Y More than 1 month to 2 months 
More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
6 of 6 standards 
Mathematics 

6 of 6 standards 

Who was involved in developing the alternate 
assessment? 

Y Assessment company 

Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 

Y Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Student’s classroom teacher 
School- or district-based educator 
State or state-contracted scorer 

Y Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Colorado 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
State assessment staff 
State instruction and curriculum staff 
•A Test vendor 
•A Outside experts 
•A Special education teachers 
•A General education teachers 
■A Content specialists 

School psychologists/counselors 
School/district/state administrators 
Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Judgmental Policy Capturing standard- 
setting methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Colorado 
proficiency levels for 
alternate assessment 


Advanced 1 




Proficient 


Novice 




Developing 




Emerging 


Basic 


Exploring 




Inconclusive 



1 State did not report label for Advanced level. 



Students who performed at the Novice and 
Developing and Emerging levels were proficient 
for purposes of the No Child Left Behind Act, 
whereas students who scored at the Exploring and 
Inconclusive levels were not proficient. 



Alternate Achievement Descriptors 

Colorado developed 

The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

•A Descriptors that were unique for each grade 
tested 

An example of the Developing descriptor in 

reading/language arts (grade 3) is as follows: 

• The Developing reader interacts with 

literacy materials purposefully. The student 
participates in literacy activities in a group, 
gaining information. The student has some 
early reading strategies, including a few sight 
words and print to picture skills. The student 
is developing language and decoding skills, 
follows simple directions and uses familiar 
vocabulary effectively, including prepositions. 
When the story is read or signed, the student 
knows what the story is about and can 
answer who, what and where questions; 
make predictions based on cause and effect; 
and use prior knowledge to relate to the 
story. 

Alternate Achievement Cut Scores 

Colorado developed 

•A Unique cut scores for each grade 
Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Colorado 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


✓ 


✓ 




✓ 


State assessment staff 


✓ 




✓ 




State instruction and curriculum staff 










Test vendor 


✓ 






✓ 


Outside experts 


✓ 


✓ 




✓ 


Special education teachers 


✓ 


✓ 


✓ 


✓ 


General education teachers 






✓ 


✓ 


Content specialists 


✓ 




✓ 


✓ 


School psychologists/counselors 










School/district/state administrators 


✓ 




✓ 




Parents 






✓ 





Validity 



Type of validity 


Types of evidence 




Scoring and reporting 
structures consistent with 
the subdomain 


Scoring and reporting 
documents 


✓ 


Relation to internal and 
external variables 


Correlational study, 
item-to-item analysis 


✓ 


Purposes of the assessment 


Survey 


✓ 


Intended and unintended 
consequences 


Survey 


✓ 


Measurement of construct 
relevance 


Construct analyses 


✓ 


Grade-level equating 


Anecdotal or 
committee process 



Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 
Grade or grade span 

Y Expanded benchmarks 

Y Alternate indicators or tasks 



Reliability 



Type of reliability 


Types of evidence 




Variability across groups 




✓ 


Internal consistency of item 
responses 


Formal study or expert 
panel review 


✓ 


Interrater consistency in 
scoring 


Statistical analysis 
conducted as part of 
actual scoring 




Standard errors of 
measurement 




Other Types of Technical Quality 


Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Formal study 


✓ 


Ongoing/monitoring 


Internal and external 
alignment studies 


✓ 


Fairness 


Regularly scheduled 
bias review by experts, 
statistical analyses 


✓ 


Implementation processes 


Training, 
administration manual/ 
guide 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Colorado 



Eligibility and Administration 



Eligibility 

Colorado provided the following guidelines to be 
used in determining when a child’s significant 
cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

Required extensive support for skill 
generalization 

•A Required modified curriculum 
Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. 

Assessment Content Selection 

State determined 

•A All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

•A No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

•A State required standardized tasks/test items/ 
rating scales 

State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

Combination of work samples and other 
evidence 

•A No student work samples 

Assessment was administered 

As part of day-to-day student instruction 
•A Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

Teacher decided content to be assessed 
Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

Teacher interpreted/recorded student 
responses or scores 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Colorado 



Scoring and Reporting 
Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

Not applicable 

• Who scored? 

Machine scored 

• How were scoring conflicts resolved? 

A third person adjudicated 
A third rater scored the assessment 

Y' One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 
Ability to generalize across settings 

Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

Y None of the above 



Individual Student Reports 

Individual student reports were given to 
S Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

S State’s achievement standards 
■S Scores 
Percentiles 

Interpretive guidance was developed for 
S School-level administrators 

Y Teachers 
S Parents 

Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
■S Scores 

Y Standard/strand breakouts 
Indicator/benchmark breakouts 

Y Performance/achievement level descriptors 
Sample test items 



Training 





Assessment 


Assessment 


r 


\ 




administrators 


scorers 1 


State department of education 




Non-face-to-face 


✓ 




website 




training 

Face-to-face 

training 


Y 




www. cd e . state . co . u s 

V 


) 



Mandatory 


Y 


training and/or 




certification 





1 Assessment was machine scored. 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Colorado 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated 

Reading/Language Arts 

Grades 




0 20 40 60 80 100 

Percent 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



in state accountability assessments 

Mathematics 




0 20 40 60 80 100 

Percent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts 

Grades , 



]87 




0 20 40 60 80 100 

Percent 



Mathematics 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.93 


0.65 


6-8 


0.86 


0.58 


9-12 


0.73 


0.45 



1 Of all students who received valid scores on the specific types of accountability assessment. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 




Connecticut 



Overview of the CMT/CAPT Skills Checklist (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 CMT/CAPT Skills 
Checklist were to 

Evaluate programs 

Y Guide classroom instruction 

Measure student progress/performance toward state standards 

Y Assess student access to content standards/general curriculum 

Y Assess individual students’ strengths/weaknesses 

Y Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

CMT/CAPT Skills Checklist 
Alternate Assessment Approaches 

Y Rating scale/checklist 
Portfolio/body of evidence 
Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 
Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

Y More than 2 weeks to 1 month 
More than 1 month to 2 months 
More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
4 of 4 standards 
Mathematics 

4 of 4 standards 

Who was involved in developing the alternate 
assessment? 

Assessment company 

Y Research company/university/independent 
researcher 

Y Technical assistance provider 

Y State personnel 
Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Y Student’s classroom teacher 
School- or district-based educator 
State or state-contracted scorer 
Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

Connecticut Alternate Assessment for the 2006-07 School Year 
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Connecticut 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

State special education staff 
•A State assessment staff 
•A State instruction and curriculum staff 
Test vendor 
•A Outside experts 
•A Special education teachers 
•A General education teachers 
•A Content specialists 

School psychologists/counselors 
•A School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Judgmental Policy Capturing standard- 
setting methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Connecticut 
proficiency levels for 
alternate assessment 


Advanced 


Independent 


Proficient 


Proficient 


Basic 


Basic 



Alternate Achievement Descriptors 

Connecticut developed 

The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

•A Descriptors that were unique for each grade 
tested 

An example of the Proficient descriptor in reading/ 
language arts is as follows: 

• Students who perform at this level complete 
academic tasks and activities derived from 
[insert grade level] content inconsistently 
and/or only with prompt support (e.g., a cue, 
a model, physical guidance, etc.). Students 
at this level demonstrate a partial ability to: 
[insert content assessed here]. 

Alternate Achievement Cut Scores 

Connecticut developed 
•A Unique cut scores for each grade 
Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Independent and 
Proficient levels were proficient for purposes of the 
No Child Left Behind Act, whereas students who 
scored at the Basic level were not proficient. 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Connecticut 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


✓ 


✓ 


✓ 


✓ 


State assessment staff 


✓ 


✓ 


✓ 


✓ 


State instruction and curriculum staff 






✓ 


✓ 


Test vendor 




✓ 




✓ 


Outside experts 


✓ 


✓ 


✓ 


✓ 


Special education teachers 


✓ 


✓ 


✓ 


✓ 


General education teachers 


✓ 


✓ 


✓ 


✓ 


Content specialists 


✓ 




✓ 


✓ 


School psychologists/counselors 










School/district/state administrators 


✓ 






✓ 


Parents 


✓ 


✓ 




✓ 



Validity 



Type of validity 


Types of evidence 




Scoring and reporting 
structures consistent with 
the subdomain 


Scoring and reporting 
documents 




Relation to internal and 
external variables 




✓ 


Purposes of the assessment 


Anecdotal or 
committee process 


✓ 


Intended and unintended 
consequences 


Not available for 
examination 


✓ 


Measurement of construct 
relevance 


Construct analyses 




Grade-level equating 





Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

General link to state content standards 
Grade or grade span 
Y Expanded benchmarks 
Alternate indicators or tasks 



Reliability 



Type of reliability 


Types of evidence 


✓ 


Variability across groups 


NCLB group 
statistical analyses 


✓ 


Internal consistency of item 
responses 


Formal study or expert 
panel review 


✓ 


Interrater consistency in 
scoring 


Statistical analysis 
conducted as part of 
training 




Standard errors of 
measurement 




Other Types of Technical Quality 


Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Formal study 


✓ 


Ongoing/monitoring 


External alignment 
study 




Fairness 


Bias review, not 
conducted regularly 


✓ 


Implementation processes 


Training 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Connecticut 



Eligibility and Administration 



Eligibility 

Connecticut provided the following guidelines 
to be used in determining when a child’s 
significant cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. 

Assessment Content Selection 

State determined 

•A All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

•A No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

•A State required standardized tasks/test items/ 
rating scales 

State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

Combination of work samples and other 
evidence 

•A No student work samples 

Assessment was administered 
•A As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

Teacher decided content to be assessed 
Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Connecticut 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

One scorer 

• Who scored? 

Student’s classroom teacher 

• How were scoring conflicts resolved? 

A third person adjudicated 
A third rater scored the assessment 

Y One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 

•Y Ability to generalize across settings 

Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Y Appropriate human and technological 
supports 

None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 


✓ 


✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 

Y Scores 
Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 

Y Scores 

Y Standard/strand breakouts 
Indicator/benchmark breakouts 

Y Performance/achievement level descriptors 
Sample test items 



( \ 

State department of education 

website 

www.sde.ct.gov 

V 1 ) 



V Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Connecticut 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Mathematics 



Grades Grades 





Percent 



Percent 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 

Reading/Language Arts Mathematics 



Grades 



3-5 


116 


|29 




170 




117 


6-8 


132 




174 




113 


9-12 


140 




174 




1 1 1 1 



0 20 40 60 80 100 



Percent 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ Students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.15 


0.28 


6-8 


0.15 


0.15 


9-12 


0.13 


0.08 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 




Delaware 



Overview of the Delaware Alternate Portfolio Assessment II (DAPA II) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Delaware Alternate 
Portfolio Assessment II (DAPA II) were to 

Y Evaluate programs 

Y Guide classroom instruction 

Y Measure student progress/performance toward state standards 

Y Assess student access to content standards/general curriculum 

Y Assess individual students’ strengths/weaknesses 

Y Document academic achievement 

Y Measure student progress toward IEP goals 



Alternate Assessment 

Delaware Alternate Portfolio Assessment II 
(DAPA II) 

Alternate Assessment Approaches 

Rating scale/checklist 

Y Portfolio/body of evidence 
Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 

Y Science 

Y Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓✓✓✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
1 of 4 standards 
Mathematics 

1 of 4 standards 

Who was involved in developing the alternate 
assessment? 

Assessment company 

Y Research company/university/independent 
researcher 

Y Technical assistance provider 

Y State personnel 

Y Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Student’s classroom teacher 
School- or district-based educator 

Y State or state-contracted scorer 
Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Delaware 



Alternate Achievement Standards 

Alternate Achievement Descriptors 

Delaware developed 1 

The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

Alternate Achievement Cut Scores 

Delaware developed 

Unique cut scores for each grade 
Unique cut scores for grade spans 
•A One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Delaware 
proficiency levels for 
alternate assessment 


Advanced 


Distinguished 




Exceeds the Standard 


Proficient 


Meets the Standard 


Basic 


Below the Standard 




Well Below the Standard 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 

•A State assessment staff 

•A State instruction and curriculum staff 

•A Test vendor 

•A Outside experts 

•A Special education teachers 

•A General education teachers 

•A Content specialists 

School psychologists/counselors 
•A School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Reasoned Judgment standard-setting 
methodology. 



Students who performed at the Distinguished, 
Exceeds the Standard, and Meets the Standard 
levels were proficient for purposes of the No Child 
Left Behind Act, whereas students who scored 
at the Below the Standard and Well Below the 
Standard levels were not proficient. 



1 State did not have descriptors. 

S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Delaware 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 1 


State special education staff 


✓ 




✓ 




State assessment staff 


✓ 


✓ 


✓ 




State instruction and curriculum staff 


✓ 




✓ 




Test vendor 


✓ 


✓ 


✓ 




Outside experts 


✓ 




✓ 




Special education teachers 


✓ 




✓ 




General education teachers 


✓ 




✓ 




Content specialists 


✓ 




✓ 




School psychologists/counselors 










School/district/state administrators 










Parents 


✓ 




✓ 





1 State did not address fairness. 



Validity 



Type of validity 



Types of evidence 



Scoring and reporting 
structures consistent with 
the subdomain 

Relation to internal and 
external variables 

Purposes of the assessment 

✓ Intended and unintended 
consequences 

✓ Measurement of construct 
relevance 

Grade-level equating 



Survey 

Anecdotal or 
committee process 



Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 






✓ 



Grade or grade span 
Expanded benchmarks 
Alternate indicators or tasks 



Reliability 


Type of reliability 


Types of evidence 


Variability across groups 




Internal consistency of item 
responses 




S Interrater consistency in 
scoring 


Statistical analysis 
conducted as part of 
training 


Standard errors of 
measurement 




Other Types of Technical Quality 


Type of technical quality 


Types of evidence 


Alignment: 




V Initial 


Anecdotal or 
committee process 


✓ Ongoing/monitoring 


Other alignment 
studies 


Fairness 




■/ Implementation processes 


Training, 
administration manual/ 
guide, monitoring 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Delaware 



Eligibility and Administration 



Eligibility 

Delaware provided the following guidelines to 
be used in determining when a child’s significant 
cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

•A Parent signature was required 

•A Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

•A Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 

•A Student work samples only 

Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 
•A As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

•A Teacher decided content to be assessed 
•A Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Delaware 



Scoring and Reporting 
Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

One scorer 

• Who scored? 

State or state-contracted scorer 

• How were scoring conflicts resolved? 

A third person adjudicated 
Y A third rater scored the assessment 

One person scored, or scores were 
combined 

Elements of student performance used in scoring 
Accuracy of student response 
•Y Ability to generalize across settings 

Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 

Y Instruction in multiple settings 

Y Opportunities to plan, monitor, and evaluate 
work 

Y Work with nondisabled peers 

Y Appropriate human and technological 
supports 

None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 


✓ 


✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 

Y Scores 
Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 

Y Scores 

Standard/strand breakouts 
Indicator/benchmark breakouts 
Performance/achievement level descriptors 
Sample test items 



( ^ 

State department of education 

website 

www.dapaonline.org 

V ) 



V Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Delaware 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Mathematics 



Grades Grades 





Percent 



Percent 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts 



Grades 



3-5 



6-8 



9-12 



154~ 



~I39~ 



H80 

I]79 

177 



~I23~ 



II 78 
H]85 



H70 



— i 1 1 1 1 

0 20 40 60 80 100 
Percent 



Mathematics 



Grades 



3-5 



6-8 



9-12 






urn 



□81 



1176 



126 



U66 



nm 



H90 



H57 



— I 1 1 1 1 

0 20 40 60 80 100 
Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ Students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.72 


0.73 


6-8 


0.85 


0.93 


9-12 


1.08 


1.14 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 




District of Columbia 



Overview of the District of Columbia Comprehensive Assessment System-Alternate 
Assessment (DC CAS-ALT) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 District of Columbia 
Comprehensive Assessment System-Alternate Assessment (DC CAS-ALT) were to 

Evaluate programs 

Y Guide classroom instruction 

Y Measure student progress/performance toward state standards 

Y Assess student access to content standards/general curriculum 
Assess individual students’ strengths/weaknesses 

Y Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

District of Columbia Comprehensive Assessment 
System-Alternate Assessment (DC CAS-ALT) 

Alternate Assessment Approaches 

Rating scale/checklist 

Y Portfolio/body of evidence 
Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 
Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
3 standards 1 
Mathematics 
3 standards 1 

Who was involved in developing the alternate 
assessment? 

Assessment company 

Y Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 
Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Student’s classroom teacher 
School- or district-based educator 

Y State or state-contracted scorer 
Machine scored 
Paraprofessional 

1 The number of general content standards varied by grade 
level. 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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District of Columbia 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

State special education staff 
•A State assessment staff 

State instruction and curriculum staff 
•A Test vendor 
•A Outside experts 
•A Special education teachers 
•A General education teachers 
■A Content specialists 

School psychologists/counselors 
School/district/state administrators 
Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Body of Work standard-setting 
methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding District of Columbia 
proficiency levels for 
alternate assessment 


Advanced 


Advanced 


Proficient 


Proficient 


Basic 


Basic 




Below Basic 



Alternate Achievement Descriptors 

District of Columbia developed 

The same descriptors that applied to all 
grades tested 

•A The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Proficient descriptor in reading/ 
language arts is as follows: 

• Provided supports such as assistive 
technology, adaptations, prompts, and / 
or modifications, and a skill that may be 
reduced in complexity and/or difficulty. 

The student demonstrates an observable 
understanding of English language arts 
content. 

Alternate Achievement Cut Scores 

District of Columbia developed 

Unique cut scores for each grade 
•A Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Advanced and 
Proficient levels were proficient for purposes of the 
No Child Left Behind Act, whereas students who 
scored at the Basic and Below Basic levels were 
not proficient. 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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District of Columbia 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


✓ 




✓ 




State assessment staff 


✓ 


✓ 


✓ 


✓ 


State instruction and curriculum staff 










Test vendor 


✓ 


✓ 


✓ 




Outside experts 


✓ 


✓ 


✓ 


✓ 


Special education teachers 


✓ 


✓ 


✓ 


✓ 


General education teachers 


✓ 


✓ 


✓ 


✓ 


Content specialists 




✓ 


V 




School psychologists/counselors 










School/district/state administrators 


✓ 


✓ 


V 


✓ 



Parents 



Validity 



Reliability 



Type of validity 


Types of evidence 




Scoring and reporting 
structures consistent with 
the subdomain 




✓ 


Relation to internal and 
external variables 


Item-to-item analysis 


✓ 


Purposes of the assessment 


Not available for 
examination 




Intended and unintended 
consequences 




✓ 


Measurement of construct 
relevance 


Construct analyses 


✓ 


Grade-level equating 


Formal study or expert 
panel review 



Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 

Y Grade or grade span 

Y Expanded benchmarks 

Y Alternate indicators or tasks 



Type of reliability 


Types of evidence 




Variability across groups 






Internal consistency of item 
responses 




✓ 


Interrater consistency in 
scoring 


Statistical analysis 
conducted as part of 
training 


✓ 


Standard errors of 
measurement 


Statistical analysis 



Other Types of Technical Quality 



Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Formal study 


✓ 


Ongoing/monitoring 


External alignment 
study 




Fairness 




✓ 


Implementation processes 


Training 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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District of Columbia 



Eligibility and Administration 



Eligibility 

District of Columbia provided the following 
guidelines to be used in determining when a child’s 
significant cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

•A Parent signature was required 

•A Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

•A Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/ tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
•A Student work samples only 

Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 
•A As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

•A Teacher decided content to be assessed 
•A Teacher selected materials 

Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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District of Columbia 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

Two scorers 

• Who scored? 

State or state-contracted scorer 

• How were scoring conflicts resolved? 

A third person adjudicated 
Y A third rater scored the assessment 

One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 
Ability to generalize across settings 

Y Amount of independence 

Y Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Y Appropriate human and technological 
supports 

None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 


✓ 


✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
Y' Scores 

Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
■S Scores 

Y Standard/strand breakouts 
Indicator/benchmark breakouts 

Y Performance/achievement level descriptors 
Sample test items 



( ^ 

State department of education 

website 

www.k12.dc.us/ 

V J 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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District of Columbia 



Participation and Proficiency (2006-07) 

Ways that students with lEPs participated in state accountability assessments 

Reading/Language Arts Mathematics 

Data not available. Data not available. 



Percentage of students who scored proficient or above on state accountability 
assessments 1 

Reading/Language Arts Mathematics 



Data not available. 



Data not available. 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Data not available. 



1 Of all students who received valid scores on the specific types of accountability assessment. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 




Florida 



Overview of the Florida Alternate Assessment Report (FAAR) (2006-07) 
Purposes of the Alternate Assessment 

(State did not have alternate achievement standards for this assessment.) 

In addition to meeting accountability requirements, the purposes of the 2006-07 Florida Alternate 
Assessment Report (FAAR) were to 

Evaluate programs 
Guide classroom instruction 

Y Measure student progress/performance toward state standards 
Assess student access to content standards/general curriculum 
Assess individual students’ strengths/weaknesses 

Y Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

Florida Alternate Assessment Report (FAAR) 
Alternate Assessment Approaches 
v" Rating scale/checklist 
Portfolio/body of evidence 
Performance task/events 
Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 

Y Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓✓✓✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

Y More than 1 month to 2 months 
More than 2 months 



Number of content standards assessed by the 
alternate assessment 1 

Reading/language arts 
Mathematics 

Who was involved in developing the alternate 
assessment? 

Assessment company 

Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 
Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Y Student’s classroom teacher 
School- or district-based educator 
State or state-contracted scorer 
Machine scored 
Paraprofessional 



1 State had four functional standards — one each for 
reading, writing, mathematics, and science. 

Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Florida 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
State assessment staff 
State instruction and curriculum staff 
Test vendor 
•A Outside experts 

Special education teachers 
General education teachers 
Content specialists 
School psychologists/counselors 
School/district/state administrators 
Parents 

Standard-Setting Methodologies 

The state did not report a standard-setting 
methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Florida 
proficiency levels for 
alternate assessment 


Advanced 


Advanced 


Proficient 


Proficient 


Basic 


Basic 



Alternate Achievement Descriptors 

Florida developed 

The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

•A Descriptors that were unique for each grade 
tested 

An example of the Proficient descriptor in reading/ 
language arts is as follows: 

• At the supported level, students are proficient 
when they can perform an activity or 
demonstrate the attainment of the indicators 
for a benchmark with some prompting. 
Further, the performance with prompting 
should occur consistently, that is, at least 
70 percent of the time. 

Alternate Achievement Cut Scores 

Florida developed 

Unique cut scores for each grade 
Unique cut scores for grade spans 
One set of cut scores for all students 

•A Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Advanced and 
Proficient levels were proficient for purposes of the 
No Child Left Behind Act, whereas students who 
scored at the Basic level were not proficient. 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Florida 



Technical Quality 1 

Who was involved in reviewing the technical quality of the alternate assessment? 

Stakeholders Validity Reliability Alignment Fairness 

State special education staff 

State assessment staff 

State instruction and curriculum staff 

Test vendor 

Outside experts 

Special education teachers 

General education teachers 

Content specialists 

School psychologists/counselors 

School/district/state administrators 

Parents 



Validity 



Reliability 



Type of validity Types of evidence 

Scoring and reporting 
structures consistent with 
the subdomain 

Relation to internal and 
external variables 

Purposes of the assessment 

Intended and unintended 
consequences 

Measurement of construct 
relevance 

Grade-level equating 



Type of reliability Types of evidence 

Variability across groups 

Internal consistency of item 
responses 

Interrater consistency in 
scoring 

Standard errors of 
measurement 



Other Types of Technical Quality 



Extended Content Standards 

The state did not develop extended content 
standards for students who were eligible for the 
alternate assessment. 



Type of technical quality Types of evidence 

Alignment: 

Initial 

Ongoing/monitoring 

Fairness 

Implementation processes 



1 State did not have an alternate assessment based on 
alternate achievement standards on which to review 
technical quality. 

S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

Florida Alternate Assessment for the 2006-07 School Year 



63 




Florida 



Eligibility and Administration 



Eligibility 

Florida provided the following guidelines to be used 
in determining when a child’s significant cognitive 
disability justified an alternate assessment based 
on alternate achievement standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

•A Parent signature was required 

Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

•A Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

•A State required standardized tasks/test items/ 
rating scales 

State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

•A Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 

As part of day-to-day student instruction 
Separately from student’s daily work 

•A Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

•A Teacher decided content to be assessed 
Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Florida 



Scoring and Reporting 
Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

One scorer 

• Who scored? 

Student’s classroom teacher 

• How were scoring conflicts resolved? 

A third person adjudicated 
A third rater scored the assessment 

Y One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 
Ability to generalize across settings 

Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

Y None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 






Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 


✓ 


✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
Y' Scores 

Percentiles 

Interpretive guidance was developed for 
School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
Y' Scores 

Standard/strand breakouts 
Indicator/benchmark breakouts 
Performance/achievement level descriptors 
Sample test items 



( \ 

State department of education 

website 

www.fldoe.org 

V ) 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Florida 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 




Percent 

□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Mathematics 




Percent 

□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts 




Percent 



Mathematics 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


1.04 


0.97 


6-8 


1.09 


1.04 


9-12 


1.49 


1.36 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 




Georgia 



Overview of the Georgia Alternate Assessment (GAA) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Georgia Alternate 
Assessment (GAA) were to 

Evaluate programs 

Y Guide classroom instruction 

Y Measure student progress/performance toward state standards 

Y Assess student access to content standards/general curriculum 

Y Assess individual students’ strengths/weaknesses 
Document academic achievement 

Measure student progress toward IEP goals 



Alternate Assessment 

Georgia Alternate Assessment (GAA) 
Alternate Assessment Approaches 
Rating scale/checklist 

Y Portfolio/body of evidence 
Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 

Y Science 

Y Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
2 standards 1 
Mathematics 

2 of 6 standards 

Who was involved in developing the alternate 
assessment? 

Y Assessment company 

Research company/university/independent 
researcher 

Y Technical assistance provider 

Y State personnel 

Y Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Student’s classroom teacher 
School- or district-based educator 

Y State or state-contracted scorer 
Machine scored 
Paraprofessional 

1 The number of general content standards varied by grade 
level. 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Georgia 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 
•A State instruction and curriculum staff 
•A Test vendor 
•A Outside experts 

Special education teachers 
•A General education teachers 
Content specialists 
School psychologists/counselors 
School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Judgmental Policy Capturing and Body of 
Work standard-setting methodologies. 



Alternate Achievement Descriptors 

Georgia developed 

•A The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Established Progress descriptor 
in reading/language arts is as follows: 

• Based on evidence in the portfolio, the 
student demonstrates an understanding of 
fundamental skills and knowledge aligned 
to grade-appropriate ELA, mathematics, 
science, or social studies standards. He/ 
she is working on and showing progress in 
academic content at an access/entry level. 
The student performs meaningful tasks using 
grade-appropriate materials. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Georgia 
proficiency levels for 
alternate assessment 


Advanced 


Extending Progress 


Proficient 


Established Progress 


Basic 


Emerging Progress 



Alternate Achievement Cut Scores 

Georgia developed 

Unique cut scores for each grade 
•A Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Extending Progress 
and Established Progress levels were proficient for 
purposes of the No Child Left Behind Act, whereas 
students who scored at the Emerging Progress 
level were not proficient. 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Georgia 



Technical Quality 



Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


✓ 


✓ 




✓ 


State assessment staff 


✓ 


✓ 




✓ 


State instruction and curriculum staff 






✓ 


✓ 


Test vendor 


✓ 


✓ 


✓ 


✓ 


Outside experts 


✓ 


✓ 




✓ 


Special education teachers 


✓ 




✓ 


✓ 


General education teachers 
Content specialists 
School psychologists/counselors 
School/district/state administrators 






✓ 


✓ 


Parents 








✓ 



Validity 



Type of validity 


Types of evidence 




Scoring and reporting 
structures consistent with 
the subdomain 

Relation to internal and 
external variables 




✓ 


Purposes of the assessment 


Not available for 
examination 


✓ 


Intended and unintended 


Post hoc data 




consequences 


collection/analysis 


✓ 


Measurement of construct 
relevance 

Grade-level equating 


Construct analyses 



Extended Content Standards 

The state did not develop extended content 
standards for students who were eligible for the 
alternate assessment. 



Reliability 



Type of reliability 


Types of evidence 


■/ Variability across groups 


Not available for 
examination 


S Internal consistency of item 
responses 


Formal study or expert 
panel review 


S Interrater consistency in 
scoring 


Statistical analysis 
conducted as part of 
actual scoring 


Standard errors of 
measurement 




Other Types of Technical Quality 


Type of technical quality 


Types of evidence 



Alignment: 



✓ 


Initial 


Anecdotal or 
committee process 


✓ 


Ongoing/monitoring 


Other alignment 
studies 


✓ 


Fairness 


Bias review, not 
conducted regularly 


✓ 


Implementation processes 


Training, 



administration manual/ 
guide, monitoring, post 
hoc data collection/ 



analysis 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Georgia 



Eligibility and Administration 



Eligibility 

Georgia provided the following guidelines to be 
used in determining when a child’s significant 
cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

•A Parent signature was required 

•A Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

•A Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
•A Student work samples only 

Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 
•A As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

•A Teacher decided content to be assessed 
•A Teacher selected materials 

Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Georgia 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

One scorer 

• Who scored? 

State or state-contracted scorer 

• How were scoring conflicts resolved? 

A third person adjudicated 
A third rater scored the assessment 

Y One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 

•Y Ability to generalize across settings 
•Y Amount of independence 

Y Amount of progress 

Environmental elements used in scoring 

Y Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Y Work with nondisabled peers 

Appropriate human and technological 
supports 

None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 




Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 


✓ 


✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 

Y Scores 
Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 

Y Scores 

Y Standard/strand breakouts 
Indicator/benchmark breakouts 

Y Performance/achievement level descriptors 
Sample test items 



( ^ 

State department of education 

website 

www.doe.kl 2.ga.us/ 

V ) 



V Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Georgia 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Mathematics 



Grades Grades 





Percent 



Percent 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 

Reading/Language Arts Mathematics 




Percent Percent 



□ Students with an IEP taking an alternate assessment □ Students with an IEP taking an alternate assessment 

based on alternate achievement standards based on alternate achievement standards 

□ students with an IEP taking a regular assessment □ Students with an IEP taking a regular assessment 

with or without accommodations with or without accommodations 

□ All students including students with lEPs □ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.63 


0.65 


6-8 


0.85 


0.86 


9-12 


0.88 


0.91 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 



Hawaii 



Overview of the Hawaii State Alternate Assessment (HSAA) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Hawaii State Alternate 
Assessment (HSAA) were to 

Y Evaluate programs 

Y Guide classroom instruction 

Y Measure student progress/performance toward state standards 

Y Assess student access to content standards/general curriculum 

Y Assess individual students’ strengths/weaknesses 
Document academic achievement 

Measure student progress toward IEP goals 



Alternate Assessment 

Hawaii State Alternate Assessment (HSAA) 
Alternate Assessment Approaches 
v" Rating scale/checklist 

Y Portfolio/body of evidence 
Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 
Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓✓✓✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
3 of 3 standards 
Mathematics 

5 of 5 standards 

Who was involved in developing the alternate 
assessment? 

Y Assessment company 

Y Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 

Y Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

Y A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Y Student’s classroom teacher 

Y School- or district-based educator 
State or state-contracted scorer 
Machine scored 

Y Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

Hawaii Alternate Assessment for the 2006-07 School Year 
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Hawaii 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 
•A State instruction and curriculum staff 
•A Test vendor 
•A Outside experts 
•A Special education teachers 
General education teachers 
Content specialists 
•A School psychologists/counselors 
•A School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Bookmark or Item Mapping standard- 
setting methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Hawaii 
proficiency levels for 
alternate assessment 


Advanced 


Exceeds Proficiency 


Proficient 


Meets Proficiency 


Basic 


Approaches Proficiency 




Well Below Proficiency 



Alternate Achievement Descriptors 

Hawaii developed 

•A The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Meets Proficiency descriptor in 
reading/language arts is as follows: 

• Student demonstrates an understanding 
of basic reading concepts when provided 
moderate support. Student demonstrates the 
basic reading skills based on the Alternate 
Performance Indicators typically at the 
Progressing level with some at the Mastered 
level. This level of functioning occurs in two 
or more settings. 

Alternate Achievement Cut Scores 

Hawaii developed 

•A Unique cut scores for each grade 
Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Exceeds 
Proficiency and Meets Proficiency levels were 
proficient for purposes of the No Child Left 
Behind Act, whereas students who scored at the 
Approaches Proficiency and Well Below Proficiency 
levels were not proficient. 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Hawaii 



Technical Quality 



Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


✓ 


✓ 


✓ 


✓ 


State assessment staff 


✓ 


✓ 


✓ 


✓ 


State instruction and curriculum staff 








✓ 


Test vendor 


✓ 


✓ 




✓ 


Outside experts 


✓ 


✓ 


✓ 


✓ 


Special education teachers 


✓ 




✓ 


✓ 


General education teachers 


✓ 




✓ 


✓ 


Content specialists 






✓ 


✓ 


School psychologists/counselors 










School/district/state administrators 










Parents 


✓ 




✓ 


✓ 



Validity Reliability 



Type of validity Types of evidence 



✓ 


Scoring and reporting 
structures consistent with 
the subdomain 


Scoring and reporting 
documents 


✓ 


Relation to internal and 
external variables 


Correlational analysis 
using external 
measures 


✓ 


Purposes of the assessment 


Survey 


✓ 


Intended and unintended 
consequences 


Survey 


✓ 


Measurement of construct 
relevance 

Grade-level equating 


Statistical analyses, 
construct analyses 



Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 

Y Grade or grade span 

Y Expanded benchmarks 

Y Alternate indicators or tasks 



Type of reliability Types of evidence 



Variability across groups 

S Internal consistency of item 
responses 

■/ Interrater consistency in 
scoring 

Standard errors of 
measurement 


Formal study or expert 
panel review 

Statistical analysis 
conducted as part of 
actual scoring 


Other Types of Technical Quality 


Type of technical quality 


Types of evidence 


Alignment: 


V Initial 


Formal study 


Ongoing/monitoring 


■/ Fairness 


Regularly scheduled 
bias review by experts 


✓ Implementation processes 


Training, 
administration manual/ 
guide, monitoring, post 
hoc data collection/ 
analysis 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Hawaii 



Eligibility and Administration 



Eligibility 

Hawaii provided the following guidelines to be used 
in determining when a child’s significant cognitive 
disability justified an alternate assessment based 
on alternate achievement standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

•A Parent signature was required 

•A Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

•A All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

•A A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

State provided instructions on types and 
amounts of evidence to be collected 

•A Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

•A Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 
•A As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

Teacher decided content to be assessed 
•A Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Hawaii 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

Three or more scorers 

• Who scored? 

Student’s classroom teacher, school- or 
district-based educator, paraprofessional 

• How were scoring conflicts resolved? 

A third person adjudicated 
Y A third rater scored the assessment 

One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 
Ability to generalize across settings 
Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

Y None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 


✓ 


✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
Y' Scores 

Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
■S Scores 

Y Standard/strand breakouts 
Indicator/benchmark breakouts 

Y Performance/achievement level descriptors 
Sample test items 



f A 

State department of education 
website 

www.doe.k12.hi.us/ 

V ) 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Hawaii 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 

Grades 




Percent 

□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Mathematics 




Percent 

□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Grades 



3-5 



6-8 



9-12 



Reading/Language Arts 




0 20 40 60 80 100 



Percent 



Mathematics 



Grades 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ Students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.20 


0.19 


6-8 


0.19 


0.17 


9-12 


0.09 


0.04 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 



Idaho 



Overview of the Idaho Alternate Assessment (IAA) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Idaho Alternate 
Assessment (IAA) were to 

Evaluate programs 

Y Guide classroom instruction 

Y Measure student progress/performance toward state standards 

Y Assess student access to content standards/general curriculum 

Y Assess individual students’ strengths/weaknesses 

Y Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

Idaho Alternate Assessment (IAA) 

Alternate Assessment Approaches 
v" Rating scale/checklist 

Y Portfolio/body of evidence 
Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 
Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
5 of 5 standards 
Mathematics 

7 of 7 standards 

Who was involved in developing the alternate 
assessment? 

Assessment company 

Y Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 

Y Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

Y A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Y Student’s classroom teacher 

Y School- or district-based educator 
State or state-contracted scorer 
Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Idaho 



Alternate Achievement Standards 

Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 

State instruction and curriculum staff 
Test vendor 
•A Outside experts 
•A Special education teachers 
General education teachers 
Content specialists 
•A School psychologists/counselors 
•A School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Bookmark or Item Mapping standard- 
setting methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Idaho 
proficiency levels for 
alternate assessment 


Advanced 


Advanced 


Proficient 


Proficient 


Basic 


Basic 




Below Basic 



Alternate Achievement Descriptors 

Idaho developed 

•A The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Proficient descriptor in reading/ 

language arts is as follows: 

• Student demonstrates developing use and 
application of alternate knowledge and 
skills and exhibits them with concrete-level 
and some abstract-level tasks. In terms of 
language: Student creates simple stories or 
journal entries with words or symbols with 
moderate adult or peer support. Listens 
to others, participates in discussions, and 
effectively expresses ideas and feelings 
using words or symbols with some adult or 
peer support. 

Alternate Achievement Cut Scores 

Idaho developed 

•A Unique cut scores for each grade 
Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Advanced and 
Proficient levels were proficient for purposes of the 
No Child Left Behind Act, whereas students who 
scored at the Basic and Below Basic levels were 
not proficient. 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Idaho 



Technical Quality 



Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


✓ 


✓ 


✓ 


✓ 


State assessment staff 




✓ 




✓ 


State instruction and curriculum staff 










Test vendor 










Outside experts 


✓ 


✓ 


✓ 


✓ 


Special education teachers 


✓ 


✓ 


✓ 


✓ 


General education teachers 
Content specialists 
School psychologists/counselors 
School/district/state administrators 
Parents 


✓ 




✓ 


✓ 



Validity Reliability 



Type of validity 


Types of evidence 


Type of reliability 


Types of evidence 


Scoring and reporting 




✓ 


Variability across groups 


NCLB group statistical 


structures consistent with 








analyses, review of 


the subdomain 








disability group results 


✓ Relation to internal and 


Correlational analysis 


✓ 


Internal consistency of item 


Formal study or expert 


external variables 


using external 




responses 


panel review 




measures, other type 
of analysis 


✓ 


Interrater consistency in 


Statistical analysis 






scoring 


conducted as part of 


■/ Purposes of the assessment 


Survey, alignment 






actual scoring 


■/ Intended and unintended 


study, analytic review 
of outcomes 


✓ 


Standard errors of 
measurement 


Statistical analysis 


Survey, post hoc data 








consequences 


collection/analysis 








■/ Measurement of construct 


Statistical analyses, 


Other Types of Technical Quality 


relevance 


construct analyses 








Grade-level equating 






Types of evidence 






lype ot technical quality 








Alignment: 




Extended Content Standards 










✓ 


Initial 


Formal study 


The state developed extended content standards 


✓ 


Ongoing/monitoring 


Internal and external 


for students who were eligible for the alternate 




alignment studies 


assessment. These extended content standards 


✓ 


Fairness 


Bias review, not 


were expressed by 








conducted regularly 


General link to state content standards 


✓ 


Implementation processes 


Training, monitoring 



■S Grade or grade span 
Y Expanded benchmarks 



Alternate indicators or tasks 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Idaho 



Eligibility and Administration 



Eligibility 

Idaho provided the following guidelines to be used 
in determining when a child’s significant cognitive 
disability justified an alternate assessment based 
on alternate achievement standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

Parent signature was required 

•/ Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

s All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

•/ No independent verification process 



Administration 

Assessment conducted by 
•/ The student’s special education teacher 

s A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

v' State required standardized tasks/test items/ 
rating scales 

•/ State provided instructions on types and 
amounts of evidence to be collected 

s Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

•/ Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 
•/ As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

Teacher decided content to be assessed 
•/ Teacher selected materials 

•/ Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

v' Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Idaho 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

One scorer 

• Who scored? 

Student’s classroom teacher, school- or 
district-based educator 

• How were scoring conflicts resolved? 

A third person adjudicated 
A third rater scored the assessment 

Y One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 

Y Ability to generalize across settings 

Y Amount of independence 

Y Amount of progress 

Environmental elements used in scoring 

Y Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Y Appropriate human and technological 
supports 

None of the above 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
Y' Scores 

Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
■S Scores 

Standard/strand breakouts 

Y Indicator/benchmark breakouts 
Performance/achievement level descriptors 
Sample test items 



Training 





Assessment 


Assessment 


r 


\ 




administrators 


scorers 


State department of education 




Non-face-to-face 


✓ 


✓ 


website 




training 

Face-to-face 

training 

Mandatory 
training and/or 
certification 


✓ 


✓ 


www.sde.idaho.gov/specialeducation 

V 


) 











Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Idaho 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Grades 



3-5 

6-8 

9-12 



92 



91 



90 



20 40 60 

Percent 



80 100 



Grades 



3-5 

6-8 

9-12 



Mathematics 



92 



91 



90 



0 20 



— i 1 — 

40 60 

Percent 



80 100 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts 




0 20 40 60 80 100 



Percent 



Mathematics 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.61 


0.57 


6-8 


0.67 


0.60 


9-12 


0.57 


0.54 



1 Of all students who received valid scores on the specific types of accountability assessment. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 




Illinois 



Overview of the Illinois Alternate Assessment (IAA) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purpose of the 2006-07 Illinois Alternate 
Assessment (IAA) was to 

Evaluate programs 
Guide classroom instruction 

Measure student progress/performance toward state standards 
Assess student access to content standards/general curriculum 
Assess individual students’ strengths/weaknesses 
Y Document academic achievement 

Measure student progress toward IEP goals 



Alternate Assessment 

Illinois Alternate Assessment (IAA) 

Alternate Assessment Approaches 
Rating scale/checklist 

Y Portfolio/body of evidence 
Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 

Y Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 

1 of 5 standards 
Mathematics 

2 of 5 standards 

Who was involved in developing the alternate 
assessment? 

Y Assessment company 

Research company/university/independent 
researcher 

Technical assistance provider 

State personnel 

Parents 

Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Student’s classroom teacher 
School- or district-based educator 

Y State or state-contracted scorer 
Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Illinois 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

State special education staff 
State assessment staff 
State instruction and curriculum staff 
Test vendor 
•A Outside experts 
•A Special education teachers 
General education teachers 
Content specialists 
School psychologists/counselors 
School/district/state administrators 
Parents 

Standard-Setting Methodologies 

The state did not report a standard-setting 
methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Illinois 
proficiency levels for 
alternate assessment 


Advanced 


Attaining 


Proficient 


Progressing 


Basic 


Emerging 




Attempting 



Alternate Achievement Descriptors 

Illinois developed 

•A The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Progressing descriptor in 
reading/language arts is as follows: 

• Individualized student work demonstrates 
moderate progress in the knowledge 
and skills in the subject through limited 
connections to the Illinois Learning 
Standards. Students exhibit a basic ability to 
generalize their knowledge and skills. 

Alternate Achievement Cut Scores 

Illinois developed 

Unique cut scores for each grade 
Unique cut scores for grade spans 
•A One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Attaining and 
Progressing levels were proficient for purposes of 
the No Child Left Behind Act, whereas students 
who scored at the Emerging and Attempting levels 
were not proficient. 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Illinois 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 










State assessment staff 










State instruction and curriculum staff 


✓ 






✓ 


Test vendor 


✓ 


✓ 




✓ 


Outside experts 


✓ 






✓ 


Special education teachers 


✓ 




✓ 


✓ 


General education teachers 


✓ 






✓ 


Content specialists 


✓ 




✓ 


✓ 


School psychologists/counselors 










School/district/state administrators 


✓ 




✓ 




Parents 


✓ 






✓ 



Validity 



Reliability 



Type of validity 


Types of evidence 


■/ Scoring and reporting 
structures consistent with 
the subdomain 


Scoring and reporting 
documents 


Relation to internal and 
external variables 




Purposes of the assessment 

Intended and unintended 
consequences 

Measurement of construct 
relevance 

Grade-level equating 





Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

General link to state content standards 
Grade or grade span 
Expanded benchmarks 
Y Alternate indicators or tasks 



Type of reliability 


Types of evidence 




Variability across groups 




✓ 


Internal consistency of item 
responses 


Formal study or expert 
panel review 


✓ 


Interrater consistency in 
scoring 


Statistical analysis 
conducted as part of 
actual scoring 




Standard errors of 
measurement 




Other Types of Technical Quality 


Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Formal study 


✓ 


Ongoing/monitoring 


External and other 
alignment studies 


✓ 


Fairness 


Bias review, not 
conducted regularly 


✓ 


Implementation processes 


Administration manual/ 
guide 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Illinois 



Eligibility and Administration 



Eligibility 

Illinois provided the following guidelines to be used 
in determining when a child’s significant cognitive 
disability justified an alternate assessment based 
on alternate achievement standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. 

Assessment Content Selection 

State determined 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

•A Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

•A No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
•A Student work samples only 

Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 
•A As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

•A Teacher decided content to be assessed 
•A Teacher selected materials 

Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

Teacher interpreted/recorded student 
responses or scores 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Illinois 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

Two scorers 

• Who scored? 

State or state-contracted scorer 

• How were scoring conflicts resolved? 

Y A third person adjudicated 

A third rater scored the assessment 

One person scored, or scores were 
combined 

Elements of student performance used in scoring 
Accuracy of student response 
Ability to generalize across settings 
Amount of independence 

Y Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

Y None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


Y 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 




✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
Y' Scores 

Percentiles 

Interpretive guidance was developed for 
School-level administrators 
Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
■S Scores 

Y Standard/strand breakouts 
Indicator/benchmark breakouts 
Performance/achievement level descriptors 
Sample test items 



( \ 

State department of education 

website 

www.isbe.state.il.us/ 

V ) 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Illinois 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Grades 



3-5 

6-8 

9-12 



93 



93 



E 



91 



0 20 



40 60 

Percent 



80 100 



Grades 



3-5 

6-8 

9-12 



Mathematics 



93 



93 



91 



0 20 



— i 1 — 

40 60 

Percent 



80 100 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 

Reading/Language Arts Mathematics 




Percent Percent 



□ Students with an IEP taking an alternate assessment □ Students with an IEP taking an alternate assessment 

based on alternate achievement standards based on alternate achievement standards 

□ students with an IEP taking a regular assessment □ Students with an IEP taking a regular assessment 

with or without accommodations with or without accommodations 

□ All students including students with lEPs □ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.64 


0.57 


6-8 


0.63 


0.54 


9-12 


0.65 


0.57 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 




Indiana 



Overview of the Indiana Standards Tool for Alternate Reporting (ISTAR) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Indiana Standards Tool 
for Alternate Reporting (ISTAR) were to 

Y Evaluate programs 

Y Guide classroom instruction 

Y Measure student progress/performance toward state standards 

Y Assess student access to content standards/general curriculum 

Y Assess individual students’ strengths/weaknesses 

Y Document academic achievement 

Y Measure student progress toward IEP goals 



Alternate Assessment 

Indiana Standards Tool for Alternate Reporting 
(ISTAR) 

Alternate Assessment Approaches 

Y Rating scale/checklist 
Portfolio/body of evidence 
Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 
Science 
Social studies 

Y Functional skills 
Grades Assessed 

✓ ✓✓✓✓✓✓✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
7 of 7 standards 
Mathematics 

7 of 7 standards 

Who was involved in developing the alternate 
assessment? 

Assessment company 

Y Research company/university/independent 
researcher 

Y Technical assistance provider 

Y State personnel 

Y Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

Y A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Y Student’s classroom teacher 
School- or district-based educator 
State or state-contracted scorer 
Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Indiana 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 
•A State instruction and curriculum staff 
Test vendor 
•A Outside experts 
•A Special education teachers 
•A General education teachers 
•A Content specialists 
•A School psychologists/counselors 
•A School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Bookmark or Item Mapping standard- 
setting methodology. 



Alternate Achievement Descriptors 

Indiana developed 

The same descriptors that applied to all 
grades tested 

•A The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Pass descriptor in reading/ 
language arts (grades 9-10) is as follows: 

• Documented strengths may include but are 
not limited to conceptual awareness and 
understanding of early reading and language 
arts skills, such as recognizing and naming 
actions or settings within books, listening and 
paying attention to others’ stories, and using 
writing utensils or communication devices as 
a way of communicating. 

Alternate Achievement Cut Scores 



Alternate Achievement Level Labels lndiana develo P ed 



Unique cut scores for each grade 



NCLB-required 


Corresponding Indiana 
proficiency levels for 


•A Unique cut scores for grade spans 


proficiency levels 


alternate assessment 


One set of cut scores for all students 


Advanced 


Pass+ 


Other approaches (e.g., applying a rubric to 


Proficient 


Pass 


determine proficiency level) 


Basic 


Basic 





Students who performed at the Pass+ and Pass 
levels were proficient for purposes of the No Child 
Left Behind Act, whereas students who scored at 
the Basic level were not proficient. 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Indiana 



Technical Quality 



Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


✓ 




✓ 


✓ 


State assessment staff 


✓ 




✓ 


✓ 


State instruction and curriculum staff 


✓ 




✓ 


✓ 


Test vendor 










Outside experts 


✓ 




✓ 


✓ 


Special education teachers 


✓ 


✓ 


✓ 


✓ 


General education teachers 


✓ 


✓ 


✓ 


✓ 


Content specialists 


✓ 


✓ 


✓ 


✓ 


School psychologists/counselors 


✓ 




✓ 




School/district/state administrators 


✓ 




✓ 




Parents 


✓ 






✓ 



Validity 



Type of validity 


Types of evidence 




Scoring and reporting 
structures consistent with 
the subdomain 


Scoring and reporting 
documents 




Relation to internal and 
external variables 




✓ 


Purposes of the assessment 


Anecdotal or 
committee process 


✓ 


Intended and unintended 
consequences 


Survey 


✓ 


Measurement of construct 
relevance 


Anecdotal or 
committee process 


✓ 


Grade-level equating 


Not available for 
examination 



Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 

Y Grade or grade span 

Y Expanded benchmarks 

Y Alternate indicators or tasks 



Reliability 



Type of reliability 


Types of evidence 




Variability across groups 






Internal consistency of item 
responses 




✓ 


Interrater consistency in 
scoring 


Statistical analysis 
conducted as part of 
actual scoring 




Standard errors of 
measurement 




Other Types of Technical Quality 


Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Formal study 


✓ 


Ongoing/monitoring 


External alignment 
study 


✓ 


Fairness 


Bias review, not 
conducted regularly 


✓ 


Implementation processes 


Training, 
administration manual/ 
guide, post hoc data 
collection/analysis 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Indiana 



Eligibility and Administration 



Eligibility 

Indiana provided the following guidelines to be 
used in determining when a child’s significant 
cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

•A Parent signature was required 

•A Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

•A All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

•A A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

State provided instructions on types and 
amounts of evidence to be collected 

•A Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

•A Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 

As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

•A Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

•A Teacher decided content to be assessed 
•A Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Indiana 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

One scorer 

• Who scored? 

Student’s classroom teacher 

• How were scoring conflicts resolved? 

A third person adjudicated 
A third rater scored the assessment 

Y One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 

Y Ability to generalize across settings 

Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

Y None of the above 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
Y' Scores 

Y Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
■S Scores 

Y Standard/strand breakouts 

Y Indicator/benchmark breakouts 

Y Performance/achievement level descriptors 
Sample test items 



Training 





Assessment 


Assessment 


r 


\ 




administrators 


scorers 


State department of education 




Non-face-to-face 


✓ 


✓ 


website 




training 






www.doe.state.in.us/ 




Face-to-face 

training 


✓ 


✓ 




) 



Mandatory 
training and/or 
certification 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Indiana 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Grades 



3-5 

6-8 

9-12 



91 



92 



86 



0 20 



40 60 

Percent 



80 100 



Grades 



3-5 

6-8 

9-12 



Mathematics 



91 



92 



86 



0 20 



— i 1 — 

40 60 

Percent 



80 100 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 

Reading/Language Arts Mathematics 




Percent Percent 



□ Students with an IEP taking an alternate assessment □ Students with an IEP taking an alternate assessment 

based on alternate achievement standards based on alternate achievement standards 

□ students with an IEP taking a regular assessment □ Students with an IEP taking a regular assessment 

with or without accommodations with or without accommodations 

□ All students including students with lEPs □ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.83 


0.81 


6-8 


0.92 


0.95 


9-12 


0.79 


0.85 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 




Iowa 



Overview of the Iowa Alternate Assessment (IAA) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting eligibility requirements, the purposes of the 2006-07 Iowa Alternate Assessment 
(IAA) were to 

Evaluate programs 
Guide classroom instruction 

•Y Measure student progress/performance toward state standards 
•Y Assess student access to content standards/general curriculum 
Assess individual students’ strengths/weaknesses 
Y Document academic achievement 

Measure student progress toward IEP goals 



Alternate Assessment 

Iowa Alternate Assessment (IAA) 

Alternate Assessment Approaches 

Y Rating scale/checklist 

Y Portfolio/body of evidence 
Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 

Y Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
1 of 1 standards 
Mathematics 

4 of 4 standards 

Who was involved in developing the alternate 
assessment? 

Assessment company 

Research company/university/independent 
researcher 

Y Technical assistance provider 

Y State personnel 
Parents 
Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Y Student’s classroom teacher 
School- or district-based educator 
State or state-contracted scorer 
Machine scored 
Paraprofessional 



■/ Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Iowa 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 
•A State instruction and curriculum staff 
Test vendor 
•A Outside experts 
•A Special education teachers 
•A General education teachers 
•A Content specialists 

School psychologists/counselors 
•A School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Judgmental Policy Capturing, Body of 
Work, and Contrasting Groups standard-setting 
methodologies. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Iowa 
proficiency levels for 
alternate assessment 


Advanced 


Advanced 


Proficient 


Proficient 


Basic 


Basic 



Alternate Achievement Descriptors 

Iowa developed 

The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

•A Descriptors that were unique for each grade 
tested 

An example of the Proficient descriptor in reading/ 
language arts is as follows: 

• Demonstrates a satisfactory understanding 
of or ability to: identify characters, setting, 
and sequence of events (plot), use context 
clues to understand some words, draw 
conclusions, identify text features, connect 
story or text to personal experiences, infer 
character feelings and traits, identify author’s 
point of view, identify main idea of passage. 

Alternate Achievement Cut Scores 

Iowa developed 

•A Unique cut scores for each grade 
Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Advanced and 
Proficient levels were proficient for purposes of the 
No Child Left Behind Act, whereas students who 
scored at the Basic level were not proficient. 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Iowa 



Technical Quality 



Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 






✓ 




State assessment staff 






✓ 


✓ 


State instruction and curriculum staff 






✓ 




Test vendor 










Outside experts 


✓ 




✓ 




Special education teachers 






✓ 


✓ 


General education teachers 






✓ 




Content specialists 






✓ 


✓ 


School psychologists/counselors 










School/district/state administrators 




✓ 


✓ 




Parents 






✓ 





Validity 



Type of validity 


Types of evidence 




Scoring and reporting 
structures consistent with 
the subdomain 


Not available for 
examination 




Relation to internal and 
external variables 






Purposes of the assessment 




✓ 


Intended and unintended 
consequences 


Survey 


✓ 


Measurement of construct 
relevance 


Anecdotal or 
committee process 


✓ 


Grade-level equating 


Not available for 
examination 



Extended Content Standards 

The state did not develop extended content 
standards for students who were eligible for the 
alternate assessment. 



Reliability 



Type of reliability 


Types of evidence 


■/ Variability across groups 


Not available for 
examination 


S Internal consistency of item 
responses 


Formal study or expert 
panel review 


S Interrater consistency in 
scoring 


Statistical analysis 
conducted as part of 
training and actual 
scoring 


Standard errors of 
measurement 




Other Types of Technical Quality 


Type of technical quality 


Types of evidence 



Alignment: 



✓ 


Initial 


Formal study 


✓ 


Ongoing/monitoring 


Internal alignment 






studies 


✓ 


Fairness 


Bias review, not 






conducted regularly 



Implementation processes 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Iowa 



Eligibility and Administration 



Eligibility 

Iowa provided the following guidelines to be used 
in determining when a child’s significant cognitive 
disability justified an alternate assessment based 
on alternate achievement standards. 

•A Had a severe cognitive disability 
Required modified instruction 

Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

Parent signature was required 

•A Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

•A Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

•A Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 
•A As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

Teacher decided content to be assessed 
•A Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Iowa 



Scoring and Reporting 
Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

Two scorers 

• Who scored? 

Student’s classroom teacher 

• How were scoring conflicts resolved? 

A third person adjudicated 
A third rater scored the assessment 

Y One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 
Ability to generalize across settings 
Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

Y None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 




✓ 


Mandatory 
training and/or 
certification 




✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
Y' Scores 

Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 
Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
Y' Scores 

Standard/strand breakouts 
Indicator/benchmark breakouts 
Performance/achievement level descriptors 
Sample test items 



( 'N 

State department of education 
website 

www.iowa.gov/educate/ 

V ) 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

Iowa Alternate Assessment for the 2006-07 School Year 



101 




Iowa 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Grades 



3-5 

6-8 

9-12 



95 



95 



95 



0 20 



40 60 

Percent 



80 100 



Grades 



3-5 

6-8 



9-12 3 



Mathematics 



95 



95 



95 



0 20 



— i 1 — 

40 60 

Percent 



80 100 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Grades 



Reading/Language Arts 

153 




9-12 



1 1 1 1 1 

0 20 40 60 80 100 
Percent 



Mathematics 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.33 


0.28 


6-8 


0.32 


0.25 


9-12 


0.28 


0.24 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 



102 



National Study on Alternate Assessments: State Profiles 



Kansas 



Overview of the Kansas Alternate Assessment (KAA) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Kansas Alternate 
Assessment (KAA) were to 

Evaluate programs 
K Guide classroom instruction 

K Measure student progress/performance toward state standards 
K Assess student access to content standards/general curriculum 
K Assess individual students’ strengths/weaknesses 
Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

Kansas Alternate Assessment (KAA) 
Alternate Assessment Approaches 
Rating scale/checklist 
K Portfolio/body of evidence 
Performance task/events 
Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 
Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
2 of 2 standards 
Mathematics 

4 of 4 standards 

Who was involved in developing the alternate 
assessment? 

Assessment company 

Y Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 

Y Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Y Student’s classroom teacher 

Y School- or district-based educator 
State or state-contracted scorer 
Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

Kansas Alternate Assessment for the 2006-07 School Year 
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Kansas 



Alternate Achievement Standards 

Who was involved in creating alternate 
achievement standards? 

•A State special education staff 

•A State assessment staff 

•A State instruction and curriculum staff 

•A Test vendor 

•A Outside experts 

•A Special education teachers 

•A General education teachers 

•A Content specialists 

School psychologists/counselors 
•A School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Body of Work standard-setting 
methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Kansas 
proficiency levels for 
alternate assessment 


Advanced 


Exemplary 




Exceeds Standard 


Proficient 


Meets Standard 


Basic 


Approaches Standard 




Academic Warning 



Students who performed at the Exemplary, 
Exceeds Standard, and Meets Standard levels 
were proficient for purposes of the No Child Left 
Behind Act, whereas students who scored at the 
Approaches Standard and Academic Warning 
levels were not proficient. 



Alternate Achievement Descriptors 

Kansas developed 

The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

•A Descriptors that were unique for each grade 
tested 

An example of the Meets Standard descriptor in 

reading/language arts (grade 3) is as follows: 

• With high levels of adult support, the 3rd 
grade student will use reading and receptive 
communication skills in the school and home 
domains. To understand text, the 3rd-grade 
student will most likely use the following 
appropriate cognitive skills (not a definitive 
list): awareness, imitation, listening, and 
responding. Across five selected assessable 
indicators chosen as appropriate for the 
individual student, a student performing at 
the meets standard level will most likely 
demonstrate an overall understanding of 
appropriate information presented to them, 
and will communicate in a variety of contexts. 
Based on the Extended Reading Standards, 
the student will: communicate receptively 
including reading and understanding text 
and respond to a variety of text, materials, 
and/or modalities. 

Alternate Achievement Cut Scores 

Kansas developed 

Unique cut scores for each grade 
Unique cut scores for grade spans 

•A One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Kansas 



Technical Quality 



Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 






✓ 




State assessment staff 


✓ 


✓ 






State instruction and curriculum staff 






✓ 




Test vendor 


✓ 


✓ 






Outside experts 


✓ 


✓ 


✓ 




Special education teachers 






✓ 


✓ 


General education teachers 






✓ 




Content specialists 






✓ 


✓ 


School psychologists/counselors 










School/district/state administrators 










Parents 






✓ 





Validity Reliability 



Type of validity Types of evidence 



✓ 


Scoring and reporting 
structures consistent with 
the subdomain 


Scoring and reporting 
documents 




Relation to internal and 
external variables 




✓ 


Purposes of the assessment 


Construct validity 
analysis 




Intended and unintended 
consequences 




✓ 


Measurement of construct 
relevance 


Construct analyses 




Grade-level equating 





Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 

Y Grade or grade span 

Y Expanded benchmarks 

Y Alternate indicators or tasks 



Type of reliability Types of evidence 



Variability across groups 

Internal consistency of item 
responses 

■/ Interrater consistency in 
scoring 

Standard errors of 
measurement 


Statistical analysis 
conducted as part of 
actual scoring 


Other Types of Technical Quality 


Type of technical quality 


Types of evidence 


Alignment: 


■/ Initial 


Anecdotal or 
committee process 


Ongoing/monitoring 


✓ Fairness 


Bias review, not 
conducted regularly 


■/ Implementation processes 


Training, 
administration manual/ 
guide, post hoc data 
collection/analysis 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

Kansas Alternate Assessment for the 2006-07 School Year 
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Kansas 



Eligibility and Administration 



Eligibility 

Kansas provided the following guidelines to be 
used in determining when a child’s significant 
cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

•A Parent signature was required 

•A Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

•A Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

•A No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

State provided instructions on types and 
amounts of evidence to be collected 

•A Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
•A Student work samples only 

Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 
•A As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

•A Teacher decided content to be assessed 
•A Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Kansas 



Scoring and Reporting 
Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

Three or more scorers 

• Who scored? 

Student’s classroom teacher, school- or 
district-based educator 

• How were scoring conflicts resolved? 

A third person adjudicated 
A third rater scored the assessment 

Y One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 
Ability to generalize across settings 
Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

Y None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 


✓ 


✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
■S Scores 

Percentiles 

Interpretive guidance was developed for 
School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
■S Scores 

Standard/strand breakouts 
Indicator/benchmark breakouts 
Performance/achievement level descriptors 
Sample test items 



C A 

State department of education 
website 

www.ksde.org 

V ) 



■/ Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Kansas 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated 

Reading/Language Arts 

Grades 




Percent 

□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 1 

□ Exempt/absent 



in state accountability assessments 

Mathematics 

Grades 




Percent 

□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 1 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 2 



Grades 



3-5 



6-8 



9-12 



Reading/Language Arts 




0 20 40 60 80 100 



Percent 



Mathematics 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.91 


0.81 


6-8 


0.77 


0.73 


9-12 


0.74 


0.79 



1 For this state this included an alternate assessment based on modified achievement standards. 

2 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 




Kentucky 



Overview of the Kentucky Alternate Assessment-Revised (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Kentucky Alternate 
Assessment-Revised were to 

Evaluate programs 
K Guide classroom instruction 

K Measure student progress/performance toward state standards 
Assess student access to content standards/general curriculum 
Assess individual students’ strengths/weaknesses 
Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

Kentucky Alternate Assessment-Revised 
Alternate Assessment Approaches 
K Rating scale/checklist 

Y Portfolio/body of evidence 
K Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 
K Reading/language arts 
K Mathematics 

Y Science 

Y Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓✓✓ 
3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
Varied by grade level 
Mathematics 

Varied by grade level 

Who was involved in developing the alternate 
assessment? 

Assessment company 

Y Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 

Y Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment ? 1 

Y Student’s classroom teacher 
School- or district-based educator 

Y State or state-contracted scorer 
Machine scored 

Paraprofess ional 

1 The scoring procedures applied only to the portfolio 
portion of the assessment. 



V Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Kentucky 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

State special education staff 
State assessment staff 
State instruction and curriculum staff 
Test vendor 
Outside experts 
•A Special education teachers 
•A General education teachers 
■A Content specialists 

School psychologists/counselors 
School/district/state administrators 
Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Performance Profile Method and the 
Body of Work standard-setting methodologies. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Kentucky 
proficiency levels for 
alternate assessment 


Advanced 


Distinguished 


Proficient 


Proficient 


Basic 


Apprentice 




Novice 



Alternate Achievement Descriptors 

Kentucky developed 

The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

•A Descriptors that were unique for each grade 
tested 

An example of the Proficient descriptor in reading/ 
language arts (grade 3) is as follows: 

• The student demonstrates an independent 
and accurate understanding of the specified 
reading skills/concepts. Occasional 
inaccuracies, which do not interfere with 
conceptual understanding, may be present. 
The student demonstrates the ability to apply 
the skilis/concepts to an authentic task and/ 
or environment. 

Alternate Achievement Cut Scores 

Kentucky developed 
•A Unique cut scores for each grade 
Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Distinguished and 
Proficient levels were proficient for purposes of the 
No Child Left Behind Act, whereas students who 
scored at the Apprentice and Novice levels were 
not proficient. 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Kentucky 



Technical Quality 



Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


✓ 


✓ 


✓ 


✓ 


State assessment staff 


✓ 


✓ 


✓ 


✓ 


State instruction and curriculum staff 


✓ 


✓ 


✓ 


✓ 



Test vendor 

Outside experts 

Special education teachers 

General education teachers 

Content specialists 

School psychologists/counselors 

School/district/state administrators 

Parents 



Validity 




Type of validity 


Types of evidence 


Scoring and reporting 
structures consistent with 
the subdomain 

Relation to internal and 
external variables 

Purposes of the assessment 

Intended and unintended 
consequences 

✓ Measurement of construct 


Statistical analyses, 


relevance 


construct analyses 


Grade-level equating 





Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

General link to state content standards 

Y Grade or grade span 

Y Expanded benchmarks 
Alternate indicators or tasks 



Reliability 


Type of reliability 


Types of evidence 


Variability across groups 




Internal consistency of item 
responses 




S Interrater consistency in 
scoring 


Statistical analysis 
conducted as part of 
actual scoring 


Standard errors of 
measurement 




Other Types of Technical Quality 


Type of technical quality 


Types of evidence 


Alignment: 




Initial 




Ongoing/monitoring 




Fairness 




✓ Implementation processes 


Administration 

manual/guide 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Kentucky 



Eligibility and Administration 



Eligibility 

Kentucky provided the following guidelines to be 
used in determining when a child’s significant 
cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. 

Assessment Content Selection 

State determined 

•A All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

•A State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
•A Student work samples only 

Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 

As part of day-to-day student instruction 
Separately from student’s daily work 

•A Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

Teacher decided content to be assessed 
•A Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Kentucky 



Scoring and Reporting 
Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

Three or more scorers 

• Who scored ?' 

Student’s classroom teacher, state or state- 
contracted scorer 

• How were scoring conflicts resolved? 

A third person adjudicated 
A third rater scored the assessment 

K One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 
Ability to generalize across settings 

K Amount of independence 

Y Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Y Appropriate human and technological 
supports 

None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 


✓ 


✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
Y' Scores 

Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
Y' Scores 

Standard/strand breakouts 
Indicator/benchmark breakouts 
Performance/achievement level descriptors 
Sample test items 



f \ 

State department of education 

website 

www.education.ky.gov 

V ) 



1 The scoring procedures applied only to the portfolio 
portion of the assessment. 

Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Kentucky 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Mathematics 



Grades Grades 





Percent 



Percent 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 

Reading/Language Arts Mathematics 





Percent 



Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.03 


0.06 


6-8 


0.04 


0.06 


9-12 


0.03 


0.10 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 



Louisiana 



Overview of the LEAP Alternate Assessment, Level 1 (LAA 1) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purpose of the 2006-07 LEAP Alternate 
Assessment, Level 1 (LAA 1) was to 

Evaluate programs 
Guide classroom instruction 

Measure student progress/performance toward state standards 
Assess student access to content standards/general curriculum 
Y Assess individual students’ strengths/weaknesses 
Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

LEAP Alternate Assessment, Level 1 (LAA 1 ) 
Alternate Assessment Approaches 
Rating scale/checklist 
Portfolio/body of evidence 

Y Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 

Y Science 

Y Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓✓✓✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

Y More than 1 month to 2 months 
More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
6 of 7 standards 
Mathematics 

4 of 6 standards 

Who was involved in developing the alternate 
assessment? 

Assessment company 

Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 

Y Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Y Student’s classroom teacher 
School- or district-based educator 
State or state-contracted scorer 
Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

Louisiana Alternate Assessment for the 2006-07 School Year 
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Louisiana 



Alternate Achievement Standards 

Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 

State instruction and curriculum staff 
•A Test vendor 
•A Outside experts 
•A Special education teachers 
General education teachers 
■A Content specialists 

School psychologists/counselors 
School/district/state administrators 
Parents 

Standard-Setting Methodologies 

The state did not report a standard-setting 
methodology. 

Alternate Achievement Level Labels 



Alternate Achievement Descriptors 

Louisiana developed 1 

The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

Alternate Achievement Cut Scores 

Louisiana developed 

Unique cut scores for each grade 
Unique cut scores for grade spans 
•A One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



NCLB-required 
proficiency levels 


Corresponding Louisiana 
proficiency levels for 
alternate assessment 


Advanced 1 




Proficient 


Proficient 


Basic 


Not Proficient 



1 State did not report label for Advanced level. 



Students who performed at the Proficient level 
were proficient for purposes of the No Child Left 
Behind Act, whereas students who scored at the 
Not Proficient level were not proficient. 



1 State did not have descriptors. 

S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Louisiana 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


✓ 




✓ 


✓ 


State assessment staff 


✓ 




✓ 


✓ 


State instruction and curriculum staff 






✓ 




Test vendor 










Outside experts 




✓ 


✓ 


✓ 


Special education teachers 


✓ 




✓ 


✓ 


General education teachers 


✓ 




✓ 


✓ 


Content specialists 






✓ 




School psychologists/counselors 










School/district/state administrators 


✓ 




✓ 




Parents 


✓ 




✓ 


✓ 



Validity 



Type of validity 


Types of evidence 




Scoring and reporting 
structures consistent with 
the subdomain 


Scoring and reporting 
documents 


✓ 


Relation to internal and 
external variables 


Other type of analysis 


✓ 


Purposes of the assessment 


Construct validity 
analysis, analytic 
review of outcomes 




Intended and unintended 
consequences 




✓ 


Measurement of construct 
relevance 


Construct analyses 




Grade-level equating 





Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 
Grade or grade span 
Expanded benchmarks 
Alternate indicators or tasks 



Reliability 



Type of reliability 


Types of evidence 


✓ 


Variability across groups 


Review of disability 
group results 


✓ 


Internal consistency of item 
responses 


Formal study or expert 
panel review 


✓ 


Interrater consistency in 
scoring 


Statistical analysis 
conducted as part of 
actual scoring 




Standard errors of 
measurement 




Other Types of Technical Quality 


Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Formal study 


✓ 


Ongoing/monitoring 


External alignment 
study 




Fairness 






Implementation processes 





S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

Louisiana Alternate Assessment for the 2006-07 School Year 
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Louisiana 



Eligibility and Administration 



Eligibility 

Louisiana provided the following guidelines to 
be used in determining when a child’s significant 
cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

•A Parent signature was required 

Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

•A All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

•A State required standardized tasks/test items/ 
rating scales 

State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

Combination of work samples and other 
evidence 

•A No student work samples 

Assessment was administered 

As part of day-to-day student instruction 
Separately from student’s daily work 

•A Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

Teacher decided content to be assessed 
Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Louisiana 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

One scorer 

• Who scored? 

Student’s classroom teacher 

• How were scoring conflicts resolved? 

A third person adjudicated 
A third rater scored the assessment 

Y One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 

•Y Ability to generalize across settings 

Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 

Y Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

None of the above 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

State’s achievement standards 

Y Scores 
Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Performance/achievement level 

Y Scores 

Standard/strand breakouts 

Y Indicator/benchmark breakouts 
Performance/achievement level descriptors 
Sample test items 



Training 





Assessment 


Assessment 


r 


\ 




administrators 


scorers 


State department of education 




Non-face-to-face 


✓ 


✓ 


website 




training 

Face-to-face 

training 


✓ 




www.doe.state.la.us/ 

V 


) 



Mandatory 


✓ 


training and/or 




certification 





v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Louisiana 



Participation and Proficiency (2006-07) 

Ways that students with lEPs participated in state accountability assessments 

Reading/Language Arts Mathematics 



Grades Grades 





Percent 



Percent 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 1 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 1 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 2 



Reading/Language Arts 




Percent 



Mathematics 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.59 


0.45 


6-8 


0.82 


0.70 


9-12 


1.66 


1.46 



1 For this state this included an alternate assessment based on modified achievement standards. 

2 Of all students who received valid scores on the specific types of accountability assessment. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 



Maine 



Overview of the Personalized Alternate Assessment Portfolio (PAAP) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Personalized Alternate 
Assessment Portfolio (PAAP) were to 

Evaluate programs 

Y Guide classroom instruction 

Y Measure student progress/performance toward state standards 
Assess student access to content standards/general curriculum 
Assess individual students’ strengths/weaknesses 

Y Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

Personalized Alternate Assessment Portfolio 
(PAAP) 

Alternate Assessment Approaches 

Rating scale/checklist 

Y Portfolio/body of evidence 
Performance task/events 

Y Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 

Y Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 

2 of 5 standards 
Mathematics 

3 of 8 standards 

Who was involved in developing the alternate 
assessment? 

Y Assessment company 

Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 
Parents 
Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Student’s classroom teacher 
School- or district-based educator 

Y State or state-contracted scorer 
Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Maine 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

State special education staff 
State assessment staff 
•A State instruction and curriculum staff 
•A Test vendor 
•A Outside experts 
•A Special education teachers 
•A General education teachers 
■A Content specialists 

School psychologists/counselors 
School/district/state administrators 
Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Body of Work standard-setting 
methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Maine 
proficiency levels for 
alternate assessment 


Advanced 


Exceeding 


Proficient 


Meeting 


Basic 


Emerging 




Attempting 



Alternate Achievement Descriptors 

Maine developed 

The same descriptors that applied to all 
grades tested 

•A The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Meeting descriptor in reading/ 
language arts is as follows: 

• Scores indicate that the student has 

consistently met the standards for the PAAP 
rubric level identified and is moving toward 
the next PAAP rubric level. 

Alternate Achievement Cut Scores 

Maine developed 

Unique cut scores for each grade 
•A Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Exceeding and 
Meeting levels were proficient for purposes of the 
No Child Left Behind Act, whereas students who 
scored at the Emerging and Attempting levels were 
not proficient. 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Maine 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 










State assessment staff 






✓ 


✓ 


State instruction and curriculum staff 








✓ 


Test vendor 


✓ 


✓ 






Outside experts 


✓ 


✓ 


✓ 


✓ 


Special education teachers 






✓ 


✓ 


General education teachers 






✓ 


✓ 


Content specialists 


✓ 


✓ 


✓ 


✓ 


School psychologists/counselors 










School/district/state administrators 










Parents 








✓ 



Validity 



Type of validity 


Types of evidence 




Scoring and reporting 
structures consistent with 
the subdomain 




✓ 


Relation to internal and 
external variables 


Not available for 
examination 


✓ 


Purposes of the assessment 

Intended and unintended 
consequences 


Anecdotal or 
committee process 


✓ 


Measurement of construct 
relevance 

Grade-level equating 


Construct analyses 



Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 

Y Grade or grade span 
Expanded benchmarks 

Y Alternate indicators or tasks 



Reliability 



Type of reliability 


Types of evidence 


✓ 


Variability across groups 


Anecdotal or 
committee process 




Internal consistency of item 
responses 




✓ 


Interrater consistency in 
scoring 


Statistical analysis 
conducted as part of 
actual scoring 




Standard errors of 
measurement 




Other Types of Technical Quality 


Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Formal study 


✓ 


Ongoing/monitoring 


Internal and external 
alignment studies 


S 


Fairness 


Bias review, not 
conducted regularly 


✓ 


Implementation processes 


Training, 
administration manual/ 
guide 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Maine 



Eligibility and Administration 



Eligibility 

Maine provided the following guidelines to be used 
in determining when a child’s significant cognitive 
disability justified an alternate assessment based 
on alternate achievement standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

Required extensive support for skill 
generalization 

Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

Parent signature was required 

•A Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

•A Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

•A No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

•A State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

•A Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
•A Student work samples only 

Combination of work samples and other 
evidence 

No student work samples 
Assessment was administered 
•A As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

•A Teacher decided content to be assessed 
•A Teacher selected materials 

Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Maine 



Scoring and Reporting 
Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

Two scorers 

• Who scored? 

State or state-contracted scorer 

• How were scoring conflicts resolved? 

A third person adjudicated 
Y A third rater scored the assessment 

One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 
Ability to generalize across settings 

Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

Y None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 




✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
Y' Scores 

Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
•Y Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 

Y Scores 

Y Standard/strand breakouts 
Indicator/benchmark breakouts 
Performance/achievement level descriptors 
Sample test items 



( ^ 

State department of education 

website 

www.maine.gov/education/index.shtml 

V ) 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Maine 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Grades 



3-5 

6-8 

9-12 



92 



89 



80 



0 20 



HU 



40 60 

Percent 



— i 1 

80 100 



Grades 



3-5 

6-8 

9-12 



Mathematics 



92 



89 



85 



— i — 
20 



ns 



40 60 

Percent 



80 100 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts Mathematics 




Percent Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.88 


0.77 


6-8 


0.87 


0.87 


9-12 


0.61 


0.76 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 




Maryland 



Overview of the Alternate Maryland School Assessment (ALT-MSA) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Alternate Maryland 
School Assessment (ALT-MSA) were to 

Evaluate programs 

Y Guide classroom instruction 

Measure student progress/performance toward state standards 

Y Assess student access to content standards/general curriculum 
Assess individual students’ strengths/weaknesses 

Y Document academic achievement 

Y Measure student progress toward IEP goals 



Alternate Assessment 

Alternate Maryland School Assessment 
(ALT-MSA) 

Alternate Assessment Approaches 

Rating scale/checklist 

Y Portfolio/body of evidence 

Y Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 
Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
3 of 3 standards 
Mathematics 

6 of 7 standards 

Who was involved in developing the alternate 
assessment? 

Y Assessment company 

Y Research company/university/independent 
researcher 

Y Technical assistance provider 

Y State personnel 

Y Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Student’s classroom teacher 
School- or district-based educator 

Y State or state-contracted scorer 
Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Maryland 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 
•A State instruction and curriculum staff 
Test vendor 
•A Outside experts 
•A Special education teachers 
General education teachers 
•A Content specialists 

School psychologists/counselors 
•A School/district/state administrators 
Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Body of Work standard-setting 
methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Maryland 
proficiency levels for 
alternate assessment 


Advanced 


Advanced 


Proficient 


Proficient 


Basic 


Basic 



Alternate Achievement Descriptors 

Maryland developed 

•A The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Proficient descriptor in reading/ 
language arts is as follows: 

• Students at this level demonstrate at least 
60% but less than 90% attainment of their 
identified mastery objectives in reading 
(attainment of 6 to 8 of the student’s Mastery 
Objectives in a given content area). The goal 
for all students is to reach the proficient or 
advanced level. 

Alternate Achievement Cut Scores 

Maryland developed 

Unique cut scores for each grade 
Unique cut scores for grade spans 
•A One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Advanced and 
Proficient levels were proficient for purposes of the 
No Child Left Behind Act, whereas students who 
scored at the Basic level were not proficient. 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Maryland 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 1 


State special education staff 


✓ 




✓ 




State assessment staff 


✓ 




✓ 




State instruction and curriculum staff 


✓ 




✓ 




Test vendor 


✓ 


✓ 


✓ 




Outside experts 


✓ 




✓ 




Special education teachers 


✓ 








General education teachers 










Content specialists 


✓ 









School psychologists/counselors 
School/district/state administrators 
Parents 

1 State did not address fairness. 

Validity Reliability 



Type of validity Types of evidence 

■/ Scoring and reporting Scoring and reporting 

structures consistent with documents 

the subdomain 
Relation to internal and 
external variables 

■/ Purposes of the assessment Survey 

Intended and unintended 
consequences 

■/ Measurement of construct Construct analyses 

relevance 

Grade-level equating 



Type of reliability Types of evidence 

Variability across groups 

Internal consistency of item 
responses 

■/ Interrater consistency in Statistical analysis 

scoring conducted as part of 

training and as part of 
actual scoring 

Standard errors of 
measurement 



Other Types of Technical Quality 



Extended Content Standards 

The state did not develop extended content 
standards for students who were eligible for the 
alternate assessment. 



Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Formal study 


✓ 


Ongoing/monitoring 


Other alignment 
studies 




Fairness 




✓ 


Implementation processes 


Training, 



administration manual/ 
guide, monitoring, post 



hoc data collection/ 
analysis 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Maryland 



Eligibility and Administration 



Eligibility 

Maryland provided the following guidelines to be 
used in determining when a child’s significant 
cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

•A Parent signature was required 

•A Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

•A Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
•A Student work samples only 

Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 

As part of day-to-day student instruction 
Separately from student’s daily work 

•A Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

■S Teacher decided content to be assessed 
■S Teacher selected materials 

■S Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Maryland 



Scoring and Reporting 
Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

Two scorers 

• Who scored? 

State or state-contracted scorer 

• How were scoring conflicts resolved? 

Y A third person adjudicated 

A third rater scored the assessment 

One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 
Ability to generalize across settings 
Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

Y None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 


✓ 


✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
Y' Scores 

Y Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 
Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
■S Scores 

Y Standard/strand breakouts 

Y Indicator/benchmark breakouts 

Y Performance/achievement level descriptors 
Sample test items 



f \ 

State department of education 
website 

www.marylandpublicschools.org 

V ) 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Maryland 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Grades 



3-5 

6-8 

9-12 



92 



89 



88 



20 40 60 

Percent 



80 100 



Grades 



3-5 

6-8 

9-12 



Mathematics 



92 



89 



88 



0 20 



— i 1 — 

40 60 

Percent 



80 100 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts Mathematics 



Grades 


Grades 






180 


181 


3-5 


159“ 3-5 


155 




H]81 


H]81 










182 


181 


6-8 


[36 6-8 


128 




ZJ72 


ZJ64 










1 78 


1 78 


9-12 


[31 9-12 


[32 




H\72 


170 




1 1 1 1 1 


1 1 1 1 



0 20 40 60 80 100 0 20 40 60 80 100 



Percent Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.73 


0.74 


6-8 


0.94 


0.91 


9-12 


0.94 


0.92 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 



Massachusetts 



Overview of the MCAS Alternate Assessment (MCAS-Alt) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 MCAS Alternate 
Assessment (MCAS-Alt) were to 

Evaluate programs 

Y Guide classroom instruction 

Y Measure student progress/performance toward state standards 

Y Assess student access to content standards/general curriculum 

Y Assess individual students’ strengths/weaknesses 

Y Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

MCAS Alternate Assessment (MCAS-Alt) 
Alternate Assessment Approaches 
Rating scale/checklist 

Y Portfolio/body of evidence 
Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 
v" Reading/language arts 

Y Mathematics 
v" Science 

Social studies 
Functional skills 
Grades Assessed 

✓ ✓✓✓✓✓✓✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 1 

Reading/language arts 
Varied by grade level 2 
Mathematics 

Varied by grade level 3 

Who was involved in developing the alternate 
assessment? 

Y Assessment company 

Y Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 
Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Y Paraprofessional 

Who scored the alternate assessment? 

Student’s classroom teacher 

Y School- or district-based educator 

Y State or state-contracted scorer 
Machine scored 
Paraprofessional 



1 Standards are referred to as strands in this state. 

2 The number of general content standards was 3. 

3 The number of general content standards was 5. 



V Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Massachusetts 



Alternate Achievement Standards 

Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 
•A State instruction and curriculum staff 
•A Test vendor 
•A Outside experts 
•A Special education teachers 
General education teachers 
■A Content specialists 

School psychologists/counselors 
School/district/state administrators 
Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Reasoned Judgment standard-setting 
methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Massachusetts 
proficiency levels for 
alternate assessment 


Advanced 1 




Proficient 


Progressing 


Basic 


Emerging 




Awareness 



1 State did not report label for Advanced level. 



Students who performed at the Progressing level 
were proficient for purposes of the No Child Left 
Behind Act, whereas students who scored at 
the Emerging and Awareness levels were not 
proficient. 



Alternate Achievement Descriptors 

Massachusetts developed 

•A The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Progressing descriptor in 
reading/language arts is as follows: 

• Student demonstrates a simple 
understanding below grade level 
expectations for a limited number of 
learning standards and core knowledge 
topics contained in the Massachusetts 
Curriculum Frameworks in the assessed 
content area. Students at this level require 
frequent prompting and assistance, and their 
performance is limited and inconsistent. 

Alternate Achievement Cut Scores 

Massachusetts developed 

Unique cut scores for each grade 
Unique cut scores for grade spans 
•A One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Massachusetts 



Technical Quality 



Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


✓ 


✓ 


✓ 


✓ 


State assessment staff 


✓ 


✓ 


✓ 


✓ 


State instruction and curriculum staff 


✓ 


✓ 


✓ 


✓ 


Test vendor 


✓ 


✓ 


✓ 


✓ 


Outside experts 


✓ 


✓ 


✓ 


✓ 


Special education teachers 


✓ 


✓ 




✓ 


General education teachers 


✓ 


✓ 


✓ 


✓ 


Content specialists 


✓ 


✓ 


✓ 


✓ 


School psychologists/counselors 
School/district/state administrators 
Parents 


✓ 

✓ 


✓ 


✓ 


✓ 



Validity 



Reliability 



Type of validity 


Types of evidence 


✓ 


Scoring and reporting 
structures consistent with 
the subdomain 


Scoring and reporting 
documents 


✓ 


Relation to internal and 
external variables 


Other type of analysis 


✓ 


Purposes of the assessment 


Survey, field/pilot tests, 
construct validity 
analysis, analytic 
review of outcomes, 
state monitoring/ 
program review 


✓ 


Intended and unintended 
consequences 


Public reports, post 
hoc data collection/ 
analysis 


✓ 


Measurement of construct 
relevance 


Construct analyses 


✓ 


Grade-level equating 


Anecdotal or 
committee process 



Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

•Y General link to state content standards 

Y Grade or grade span 

Y Expanded benchmarks 

Y Alternate indicators or tasks 



Type of reliability Types of evidence 



Variability across groups 

Internal consistency of item 
responses 

■/ Interrater consistency in 
scoring 

Standard errors of 
measurement 


Statistical analysis 
conducted as part of 
training and actual 
scoring 


Other Types of Technical Quality 


Type of technical quality 


Types of evidence 


Alignment: 


✓ Initial 


Formal study 


■/ Ongoing/monitoring 


Internal and external 
alignment studies 


Fairness 


■/ Implementation processes 


Training, 
administration manual/ 
guide, monitoring, post 
hoc data collection/ 
analysis 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Massachusetts 



Eligibility and Administration 



Eligibility 

Massachusetts provided the following guidelines 
to be used in determining when a child’s 
significant cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

•A Parent signature was required 

•A Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

•A Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

•A Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
•A Student work samples only 

Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 
•A As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

•A Teacher decided content to be assessed 
•A Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Massachusetts 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 1 
One scorer 

• Who scored? 

School- or district-based educator, state or 
state-contracted scorer 

• How were scoring conflicts resolved? 

Y A third person adjudicated 

A third rater scored the assessment 

One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 

•Y Ability to generalize across settings 
•Y Amount of independence 

Y Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Y Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

None of the above 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 

Y Scores 

Y Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 

Y Scores 

Y Standard/strand breakouts 
Indicator/benchmark breakouts 

Y Performance/achievement level descriptors 
Sample test items 



Training 





Assessment 


Assessment 


r 


\ 




administrators 


scorers 


State department of education 




Non-face-to-face 


✓ 




website 




training 

Face-to-face 

training 


✓ 


✓ 


www.doe.mass.edu 

V 


) 



Mandatory 


✓ 


training and/or 




certification 





1 Portion of sample doubled scored. 

S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Massachusetts 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Grades 



3-5 

6-8 

9-12 



91 



91 



87 



HE 



20 40 60 

Percent 



80 100 



Grades 



3-5 

6-8 



9-12 a 



Mathematics 



91 



91 



85 



0 20 



— i 1 — 

40 60 

Percent 



IS 



80 100 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 1 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 1 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 2 



Reading/Language Arts 




Percent 



Mathematics 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


1.24 


1.30 


6-8 


1.04 


1.08 


9-12 


0.68 


0.69 



1 For this state this included an alternate assessment based on grade-level achievement standards. 

2 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 




Michigan 



Overview of Mi-Access Functional Independence (FI) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Mi-Access Functional 
Independence (FI) were to 

Evaluate programs 
Guide classroom instruction 

Y Measure student progress/performance toward state standards 
Assess student access to content standards/general curriculum 

Y Assess individual students’ strengths/weaknesses 

Y Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

Mi-Access Functional Independence (FI) 
Alternate Assessment Approaches 
Rating scale/checklist 
Portfolio/body of evidence 
Performance task/events 

Y Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 
Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

Y More than 1 month to 2 months 
More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
4 of 4 standards 
Mathematics 

Varied by grade level 1 

Who was involved in developing the alternate 
assessment? 

Y Assessment company 

Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 

Y Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

Y A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Student’s classroom teacher 
School- or district-based educator 
State or state-contracted scorer 

Y Machine scored 
Paraprofessional 

1 The number of general content standards was 5. 



V Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Michigan 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 

•A State assessment staff 

•A State instruction and curriculum staff 

•A Test vendor 

•A Outside experts 

•A Special education teachers 

•A General education teachers 

•A Content specialists 

•A School psychologists/counselors 

•A School/district/state administrators 

•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Bookmark or Item Mapping standard- 
setting methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Michigan 
proficiency levels for 
FI alternate assessment 


Advanced 


Surpassed the 
Performance Standard 


Proficient 


Attained the 
Performance Standard 


Basic 


Emerging Toward the 
Performance Standard 



Students who performed at the Surpassed 
the Performance Standard and Attained the 
Performance Standard levels were proficient for 
purposes of the No Child Left Behind Act, whereas 
students who scored at the Emerging Toward the 
Performance Standard level were not proficient. 



Alternate Achievement Descriptors 

Michigan developed 

The same descriptors that applied to all 
grades tested 

•A The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Attained the Performance 
Standard descriptor in mathematics (grades 3-4) is 
as follows: 

• Students who attained the performance 
standard should typically be able to 

- Demonstrate basic understanding of 
numeration skills. 

- Demonstrate basic knowledge of 
measurement concepts and instruments. 

- Demonstrate basic skill in identifying, 
describing, and comparing basic 
geometric shapes and the relative 
positions of objects. 

- Demonstrate a basic ability to collect, 
organize, and summarize data. 

Alternate Achievement Cut Scores 

Michigan developed 
•A Unique cut scores for each grade 
Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Michigan 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 






✓ 


✓ 


State assessment staff 


✓ 


✓ 


✓ 


✓ 


State instruction and curriculum staff 


✓ 


✓ 


✓ 




Test vendor 


✓ 


✓ 


✓ 




Outside experts 


✓ 


✓ 


✓ 


✓ 


Special education teachers 


✓ 




✓ 


✓ 


General education teachers 


✓ 




✓ 


✓ 


Content specialists 


✓ 




✓ 


✓ 


School psychologists/counselors 






✓ 




School/district/state administrators 


✓ 




✓ 




Parents 


✓ 




✓ 


✓ 



Validity Reliability 



Type of validity Types of evidence 



✓ 


Scoring and reporting 
structures consistent with 
the subdomain 


Scoring and reporting 
documents 


✓ 


Relation to internal and 
external variables 


Item-to-item analysis 


✓ 


Purposes of the assessment 


Anecdotal or 
committee process 




Intended and unintended 
consequences 




✓ 


Measurement of construct 
relevance 


Construct analyses 




Grade-level equating 





Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 

Y Grade or grade span 

Y Expanded benchmarks 

Y Alternate indicators or tasks 



Type of reliability Types of evidence 



✓ 


Variability across groups 


NCLB group 
statistical analyses 


✓ 


Internal consistency of item 
responses 

Interrater consistency in 
scoring 


Formal study or expert 
panel review 


✓ 


Standard errors of 
measurement 


Statistical analysis 


Other Types of Technical Quality 


Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Formal study 


✓ 


Ongoing/monitoring 


External alignment 
study 


✓ 


Fairness 


Regularly scheduled 
bias review by experts, 
statistical analyses 


✓ 


Implementation processes 


Training, 
administration manual/ 
guide 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Michigan 



Eligibility and Administration 



Eligibility 

Michigan provided the following guidelines to be 
used in determining when a child’s significant 
cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

Parent signature was required 

•A Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

•A All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

•A No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

•A A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

•A State required standardized tasks/test items/ 
rating scales 

State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

Combination of work samples and other 
evidence 

•A No student work samples 

Assessment was administered 

As part of day-to-day student instruction 
•A Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

Teacher decided content to be assessed 
Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Michigan 



Scoring and Reporting 
Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

One scorer 

• Who scored? 

Machine scored 

• How were scoring conflicts resolved? 

A third person adjudicated 
A third rater scored the assessment 

Y One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 
Ability to generalize across settings 
Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Y Appropriate human and technological 
supports 

None of the above 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
Y' Scores 

Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
■S Scores 

Y Standard/strand breakouts 

Y Indicator/benchmark breakouts 
Performance/achievement level descriptors 
Sample test items 



Training 





Assessment 


Assessment 


r 


\ 




administrators 


scorers 1 


State department of education 




Non-face-to-face 


✓ 




website 




training 






www.michigan.gov/mi-access 




Face-to-face 


Y 








training 






V 


) 



Mandatory 


✓ 


training and/or 




certification 





1 Assessment was machine scored. 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Michigan 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Mathematics 



Grades Grades 





Percent 



Percent 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts 




Percent 



Mathematics 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


1.76 


1.70 


6-8 


1.72 


2.53 


9-12 


1.91 


3.02 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. Data are unreliable for all 
students including students with lEPs in grades 9-12 who scored proficient or above on state accountability assessments in 
reading/language arts. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 



Michigan 



Overview of Participation and Supported Independence (P/SI) 2006-2007 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-2007 Participation and 
Supported Independence (P/SI) were to 

Evaluate programs 
Guide classroom instruction 

Y Measure student progress/performance toward state standards 
Assess student access to content standards/general curriculum 

Y Assess individual students’ strengths/weaknesses 

Y Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

Participation and Supported Independence (P/SI) 
Alternate Assessment Approaches 
Rating scale/checklist 
Portfolio/body of evidence 

Y Performance task/events 

Y Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 
Science 
Social studies 
Functional skills 

Grades Assessed 

S V V V S V ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

Y More than 1 month to 2 months 
More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
4 of 4 standards 
Mathematics 

Varied by grade level 1 

Who was involved in developing the alternate 
assessment? 

Assessment company 

Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 

Y Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

Y A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Y Student’s classroom teacher 

Y School- or district-based educator 
State or state-contracted scorer 

Y Machine scored 

Y Paraprofessional 

1 The number of general content standards was 5. 



V Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Michigan 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 
•A State instruction and curriculum staff 
Test vendor 
•A Outside experts 
•A Special education teachers 
•A General education teachers 
•A Content specialists 
•A School psychologists/counselors 
•A School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Bookmark or Item Mapping standard- 
setting methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Michigan 
proficiency levels for 
P/SI alternate assessment 


Advanced 


Surpassed the 
Performance Standard 


Proficient 


Attained the 
Performance Standard 


Basic 


Emerging Toward the 
Performance Standard 



Students who performed at the Surpassed 
the Performance Standard and Attained the 
Performance Standard levels were proficient for 
purposes of the No Child Left Behind Act, whereas 
students who scored at the Emerging Toward the 
Performance Standard level were not proficient. 



Alternate Achievement Descriptors 

Michigan developed 

The same descriptors that applied to all 
grades tested 

•A The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Attained the Performance 

Standard descriptor in reading/language arts is as 

follows: 

• A Participation elementary student at the 
attained level in Accessing Information: 

Word Study can recognize and demonstrate 
basic understanding of some frequently 
encountered objects and/or pictures paired 
with words. Supported Independence 
students at the same performance level 
in the same component can recognize 
some frequently encountered/personally 
meaningful words (e.g., name, address, 
family members), functional words (e.g., 
exit, danger), and content area specific 
vocabulary, and demonstrate understanding 
of some functional words/symbols (e.g., exit, 
danger). 

Alternate Achievement Cut Scores 

Michigan developed 

•A Unique cut scores for each grade 
Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Michigan 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 










State assessment staff 


✓ 


✓ 


✓ 


✓ 


State instruction and curriculum staff 


✓ 


✓ 


✓ 




Test vendor 


✓ 


✓ 


✓ 


✓ 


Outside experts 


✓ 


✓ 


✓ 


✓ 


Special education teachers 


✓ 




✓ 


✓ 


General education teachers 


✓ 




✓ 


✓ 


Content specialists 


✓ 




✓ 


✓ 


School psychologists/counselors 






✓ 




School/district/state administrators 


✓ 




✓ 




Parents 


✓ 




✓ 


✓ 



Validity 



Type of validity 


Types of evidence 




Scoring and reporting 
structures consistent with 
the subdomain 


Scoring and reporting 
documents 


✓ 


Relation to internal and 
external variables 


Item-to-item analysis 


✓ 


Purposes of the assessment 


Not available for 
examination 




Intended and unintended 
consequences 




✓ 


Measurement of construct 
relevance 


Statistical analyses 




Grade-level equating 





Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 

Y Grade or grade span 

Y Expanded benchmarks 

Y Alternate indicators or tasks 



Reliability 



Type of reliability 


Types of evidence 


✓ 


Variability across groups 


NCLB group statistical 
analyses 




Internal consistency of item 
responses 




✓ 


Interrater consistency in 
scoring 


Statistical analysis 
conducted as part of 
actual scoring 


✓ 


Standard errors of 
measurement 


Statistical analysis 


Other Types of Technical Quality 


Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Formal study 


✓ 


Ongoing/monitoring 


Other alignment 
studies 


✓ 


Fairness 


Regularly scheduled 
bias review by experts 


✓ 


Implementation processes 


Training, 
administration manual/ 
guide 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Michigan 



Eligibility and Administration 



Eligibility 

Michigan provided the following guidelines to be 
used in determining when a child’s significant 
cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

Parent signature was required 

•A Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

•A All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

•A An observer/monitor who was present 

A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

•A A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

•A State required standardized tasks/test items/ 
rating scales 

State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

Combination of work samples and other 
evidence 

•A No student work samples 

Assessment was administered 

As part of day-to-day student instruction 
Separately from student’s daily work 

•A Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

Teacher decided content to be assessed 
Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Michigan 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

Two scorers 

• Who scored? 

Student’s classroom teacher, school- or 
district-based educator, machine scored, 
paraprofessional 

• How were scoring conflicts resolved? 

A third person adjudicated 
A third rater scored the assessment 

Y One person scored, or scores were 
combined 

Elements of student performance used in scoring 
•Y Accuracy of student response 
Ability to generalize across settings 

Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

Y None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 




Mandatory 
training and/or 
certification 


✓ 





Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 

Y Scores 
Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 

Y Scores 

Y Standard/strand breakouts 

Y Indicator/benchmark breakouts 
Performance/achievement level descriptors 

Y Sample test items 



( ^ 

State department of education 

website 

www.michigan.gov/mi-access 

V J 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Michigan 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Mathematics 



Grades Grades 





Percent 



Percent 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts 




Percent 



Mathematics 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


1.76 


1.70 


6-8 


1.72 


2.53 


9-12 


1.91 


3.02 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. Data are unreliable for all 
students including students with lEPs in grades 9-12 who scored proficient or above on state accountability assessments in 
reading/language arts. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 



Minnesota 



Overview of the Minnesota Test of Academic Skills (MTAS) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Minnesota Test of 
Academic Skills (MTAS) were to 

Evaluate programs 

Y Guide classroom instruction 

Y Measure student progress/performance toward state standards 

Y Assess student access to content standards/general curriculum 
Assess individual students’ strengths/weaknesses 
Document academic achievement 

Measure student progress toward IEP goals 



Alternate Assessment 

Minnesota Test of Academic Skills (MTAS) 
Alternate Assessment Approaches 
Rating scale/checklist 
Portfolio/body of evidence 

Y Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 
Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

Y More than 1 month to 2 months 
More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
1 of 3 standards 
Mathematics 
4 standards 1 

Who was involved in developing the alternate 
assessment? 

Y Assessment company 

Y Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 
Parents 
Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

Y A certified educator who was not the student’s 
teacher 

Y Paraprofessional 

Who scored the alternate assessment? 

Y Student’s classroom teacher 

Y School- or district-based educator 
State or state-contracted scorer 
Machine scored 

Y Paraprofessional 

1 The number of general content standards varied by grade 
level. 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Minnesota 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

State special education staff 
•A State assessment staff 

State instruction and curriculum staff 
•A Test vendor 
•A Outside experts 
•A Special education teachers 
•A General education teachers 
•A Content specialists 

School psychologists/counselors 
•A School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Modified Angoff and Bookmark or Item 
Mapping standard-setting methodologies. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Minnesota 
proficiency levels for 
alternate assessment 


Advanced 


Exceeds Expectations 


Proficient 


Meets Expectations 


Basic 


Partially Meets Expectations 



Students who performed at the Exceeds 
Expectations and Meets Expectations levels were 
proficient for purposes of the No Child Left Behind 
Act, whereas students who scored at the Partially 
Meets Expectations level were not proficient. 



Alternate Achievement Descriptors 

Minnesota developed 

The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

•A Descriptors that were unique for each grade 
tested 

An example of a Meets Expectations descriptor 
level in reading/language arts (Grade 3) is as 
follows: 

• Students at this level succeed at some 
of the skills represented by the alternate 
achievement standards set for the Minnesota 
Academic Standards in reading. Some 
of the skills these students demonstrate 
consistently include the following: 

- Vocabulary expansion skills such as 
identifying opposites; using direct and 
implied context clues to define literal 
meanings of specified vocabulary words 

- Comprehension skills such as identifying 
main idea and two supporting details of 
fiction text; answering and asking literal 
questions based on fiction text; retelling 
significant events in correct sequence; 
completing three-step written directions 
in sequence; identifying main idea and 
supporting details of nonfiction text; 
answering and asking literal questions 
based on nonfiction text 

- Literature skills such as identifying 
character, setting, and plot; identifying 
rhyming words; identifying similes; 
comparing and contrasting reactions to 
text 



Alternate Achievement Cut Scores 

Minnesota developed 
•A Unique cut scores for each grade 
Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 

S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Minnesota 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


✓ 






✓ 


State assessment staff 


✓ 


✓ 


✓ 


✓ 


State instruction and curriculum staff 










Test vendor 


✓ 


✓ 


✓ 




Outside experts 


✓ 


✓ 


✓ 


✓ 


Special education teachers 


✓ 




✓ 


✓ 


General education teachers 


✓ 




✓ 


✓ 


Content specialists 


✓ 




✓ 




School psychologists/counselors 










School/district/state administrators 










Parents 






✓ 


✓ 



Validity 



Type of validity 


Types of evidence 


✓ 


Scoring and reporting 
structures consistent with 
the subdomain 


Scoring and reporting 
documents 


✓ 


Relation to internal and 
external variables 


Correlational study 




Purposes of the assessment 

Intended and unintended 
consequences 




✓ 


Measurement of construct 
relevance 


Construct analyses 




Grade-level equating 





Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 

Y Grade or grade span 

Y Expanded benchmarks 
Alternate indicators or tasks 



Reliability 



Type of reliability 


Types of evidence 




Variability across groups 




✓ 


Internal consistency of item 
responses 


Formal study or expert 
panel review 


✓ 


Interrater consistency in 
scoring 


Statistical analysis 
conducted as part of 
actual scoring 


✓ 


Standard errors of 
measurement 


Statistical analysis 


Other Types of Technical Quality 


Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Formal study 




Ongoing/monitoring 




✓ 


Fairness 


Regularly scheduled 
bias review by experts 


✓ 


Implementation processes 


Monitoring, post 
hoc data collection/ 
analysis 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Minnesota 



Eligibility and Administration 



Eligibility 

Minnesota provided the following guidelines 
to be used in determining when a child’s 
significant cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

•A Parent signature was required 

•A Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

•A All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

•A A certified educator who was not the 
student’s teacher 

•A Paraprofessional 

Procedures followed in gathering performance 
evidence included 

•A State required standardized tasks/test items/ 
rating scales 

State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

Combination of work samples and other 
evidence 

•A No student work samples 

Assessment was administered 

As part of day-to-day student instruction 
•A Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

Teacher decided content to be assessed 
Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Minnesota 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

One scorer 

• Who scored? 

Student’s classroom teacher, school- or 
district-based educator, paraprofessional 

• How were scoring conflicts resolved? 

A third person adjudicated 
A third rater scored the assessment 

Y One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 
Ability to generalize across settings 

Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

Y None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 




Mandatory 
training and/or 
certification 


✓ 





Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
Y' Scores 

Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 
Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
■S Scores 

Standard/strand breakouts 

Y Indicator/benchmark breakouts 

Y Performance/achievement level descriptors 
Sample test items 



( ^ 

State department of education 

website 

http ://ed ucation . state, mn . us/ 

V ) 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Minnesota 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 




Percent 

□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Mathematics 




Percent 

□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts 




Percent 



Mathematics 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


1.05 


1.02 


6-8 


1.09 


1.00 


9-12 


1.01 


1.02 



1 Of all students who received valid scores on the specific types of accountability assessment. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 



Mississippi 



Overview of the Mississippi Alternate Assessment of Extended Curriculum 
Frameworks (MAAECF) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Mississippi Alternate 
Assessment of Extended Curriculum Frameworks (MAAECF) were to 

Evaluate programs 
Guide classroom instruction 

Y Measure student progress/performance toward state standards 
Assess student access to content standards/general curriculum 

Y Assess individual students’ strengths/weaknesses 

Y Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

Mississippi Alternate Assessment of Extended 
Curriculum Frameworks (MAAECF) 

Alternate Assessment Approaches 

Y Rating scale/checklist 

Y Portfolio/body of evidence 
Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 
Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
4 of 4 standards 
Mathematics 

Varied by grade level 

Who was involved in developing the alternate 
assessment? 

Assessment company 

Y Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 
Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Y Student’s classroom teacher 

Y School- or district-based educator 
State or state-contracted scorer 
Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Mississippi 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
State assessment staff 
State instruction and curriculum staff 
•A Test vendor 
•A Outside experts 
•A Special education teachers 
General education teachers 
■A Content specialists 

School psychologists/counselors 
School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Bookmark or Item Mapping standard- 
setting methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Mississippi 
proficiency levels for 
alternate assessment 


Advanced 


Advanced 


Proficient 


Proficient 


Basic 


Basic 




Minimal 



Alternate Achievement Descriptors 

Mississippi developed 

•A The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Proficient descriptor in reading/ 
language arts is as follows: 

• Student demonstrates the ability to 

communicate ideas when provided moderate 
support. The student’s understanding of 
basic concepts and performance of many 
reading and writing skills are inconsistent, 
but typically at the Progressing level across 
two or more settings. 

Alternate Achievement Cut Scores 

Mississippi developed 
•A Unique cut scores for each grade 
Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Advanced and 
Proficient levels were proficient for purposes of the 
No Child Left Behind Act, whereas students who 
scored at the Basic and Minimal levels were not 
proficient. 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Mississippi 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


✓ 


✓ 




✓ 


State assessment staff 


✓ 


✓ 




✓ 


State instruction and curriculum staff 


✓ 


✓ 




✓ 


Test vendor 


✓ 


✓ 


✓ 


✓ 


Outside experts 


✓ 


✓ 




✓ 


Special education teachers 


✓ 


✓ 


✓ 


✓ 


General education teachers 
Content specialists 
School psychologists/counselors 
School/district/state administrators 
Parents 


✓ 


✓ 


✓ 


✓ 



Validity 




Reliability 




Type of validity 


Types of evidence 


Type of reliability 


Types of evidence 




Scoring and reporting 




✓ 


Variability across groups 


NCLB group 




structures consistent with 








statistical analyses 




the subdomain 




✓ 


Internal consistency of item 


Formal study or expert 


✓ 


Relation to internal and 


Correlational analysis 




responses 


panel review 




external variables 


using external 


✓ 


Interrater consistency in 


Statistical analysis 






measures 




scoring 


conducted as part of 


✓ 


Purposes of the assessment 


Alignment study, 






actual scoring 






construct validity 


✓ 


Standard errors of 


Statistical analysis 






analysis 




measurement 




✓ 


Intended and unintended 


Survey 










consequences 




Other Types of Technical Quality 


✓ 


Measurement of construct 


Statistical analyses 










relevance 












Grade-level equating 




Type of technical quality 


Types of evidence 



Alignment: 



Extended Content Standards 


✓ 


Initial 


Formal study 


The state developed extended content standards 


✓ 


Ongoing/monitoring 


Other alignment 
studies 


for students who were eligible for the alternate 
assessment. These extended content standards 




Fairness 


Regularly scheduled 
bias review by experts 


were expressed by 


✓ 


Implementation processes 


Training, 



•S General link to state content standards administration 

manual/guide 

Grade or grade span 

Expanded benchmarks 
Alternate indicators or tasks 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Mississippi 



Eligibility and Administration 



Eligibility 

Mississippi provided the following guidelines 
to be used in determining when a child’s 
significant cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. 

Assessment Content Selection 

State determined 

•A All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

•A No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

•A Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 
•A As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

Teacher decided content to be assessed 
•A Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Mississippi 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

Two scorers 

• Who scored? 

Student’s classroom teacher, school- or 
district-based educator 

• How were scoring conflicts resolved? 

A third person adjudicated 
Y A third rater scored the assessment 

One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 

Y Ability to generalize across settings 

Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

Y None of the above 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
Y' Scores 

Percentiles 

Interpretive guidance was developed for 
School-level administrators 
Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
■S Scores 

Y Standard/strand breakouts 
Indicator/benchmark breakouts 

Y Performance/achievement level descriptors 
Sample test items 



Training 





Assessment 


Assessment 


r 


\ 




administrators 


scorers 


State department of education 




Non-face-to-face 


✓ 


✓ 


website 




training 

Face-to-face 

training 


✓ 


✓ 


www.mde.k12.ms.us/maaecf/ 

V 


) 



Mandatory 


✓ 


✓ 


training and/or 






certification 







Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Mississippi 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Grades 



3-5 

6-8 

9-12 



El 


89 


0 


3 


ifl 


87 


II 


0 




2 




98 


u 


1 1 1 1 1 



0 20 



40 60 

Percent 



80 100 



Mathematics 



Grades 



3-5 

6-8 

9-12 



El 


89 


1 










87 


1 


0 




2 




98 


II 


1 


1 1 


1 1 



0 20 40 60 

Percent 



80 100 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts 




Percent 



Mathematics 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.97 


0.88 


6-8 


1.02 


0.92 


9-12 


M 


M 



M Missing. Information not provided by state. 

1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 



Missouri 



Overview of the Missouri Assessment Prog ram -Alternate (MAP-A) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Missouri Assessment 
Program-Alternate (MAP-A) were to 

Y Evaluate programs 

Y Guide classroom instruction 

Y Measure student progress/performance toward state standards 

Y Assess student access to content standards/general curriculum 

Y Assess individual students’ strengths/weaknesses 

Y Document academic achievement 

Y Measure student progress toward IEP goals 



Alternate Assessment 

Missouri Assessment Program-Alternate (MAP-A) 
Alternate Assessment Approaches 
Rating scale/checklist 

Y Portfolio/body of evidence 
Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 
Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

Y More than 1 month to 2 months 
More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
2 of 7 standards 
Mathematics 

2 of 6 standards 

Who was involved in developing the alternate 
assessment? 

Y Assessment company 

Y Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 

Y Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

Y A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Y Student’s classroom teacher 
School- or district-based educator 

Y State or state-contracted scorer 
Machine scored 
Paraprofessional 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

Missouri Alternate Assessment for the 2006-07 School Year 
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Missouri 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 
•A State instruction and curriculum staff 
Test vendor 
•A Outside experts 
•A Special education teachers 
•A General education teachers 
•A Content specialists 
•A School psychologists/counselors 
•A School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Body of Work standard-setting 
methodology. 



Alternate Achievement Descriptors 

Missouri developed 

The same descriptors that applied to all 
grades tested 

•A The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Proficient descriptor in reading/ 
language arts (grades 3 to 5) is as follows: 

• Student has some understanding of the 
concepts contained in the grade appropriate 
APIs within the standards of the Reading 
Development and Processes and Standard 
English Conventions. Student work may be 
connected to the standards and demonstrate 
application. Student likely requires some 
verbal, visual and/or physical task-specific 
assistance in order to demonstrate 
knowledge of these concepts. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Missouri 
proficiency levels for 
alternate assessment 


Advanced 


Advanced 


Proficient 


Proficient 


Basic 


Basic 




Below Basic 




Level not Determined 



Alternate Achievement Cut Scores 

Missouri developed 

Unique cut scores for each grade 
•A Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Advanced and 
Proficient levels were proficient for purposes of 
the No Child Left Behind Act, whereas students 
who scored at Basic, Below Basic, and Level not 
Determined were not proficient. 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Missouri 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


✓ 




✓ 


✓ 


State assessment staff 


✓ 


✓ 


✓ 


✓ 


State instruction and curriculum staff 


✓ 




✓ 




Test vendor 


✓ 


✓ 


✓ 


✓ 


Outside experts 


✓ 


✓ 


✓ 




Special education teachers 


✓ 




✓ 


✓ 


General education teachers 


✓ 




✓ 


✓ 


Content specialists 


✓ 




✓ 


✓ 


School psychologists/counselors 










School/district/state administrators 


✓ 






✓ 


Parents 


✓ 






✓ 



Validity Reliability 



Type of validity 


Types of evidence 


Type of reliability 


Types of evidence 


Scoring and reporting 
structures consistent with 
the subdomain 

Relation to internal and 
external variables 

■/ Purposes of the assessment 
■/ Intended and unintended 


Construct validity 
analysis 

Not available for 


Variability across groups 

Internal consistency of item 
responses 

■/ Interrater consistency in 
scoring 

✓ Standard errors of 
measurement 


Statistical analysis 
conducted as part of 
actual scoring 

Statistical analysis 


consequences 

■/ Measurement of construct 
relevance 

Grade-level equating 


examination 
Statistical analyses 


Other Types of Technical Quality 








Extended Content Standards 




Type of technical quality 


Types of evidence 




Alignment: 




The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

General link to state content standards 


Initial 

•/ Ongoing/monitoring 
✓ Fairness 

Implementation processes 


Internal alignment 
studies 

Statistical analyses 



Grade or grade span 

Y Expanded benchmarks 

Y Alternate indicators or tasks 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Missouri 



Eligibility and Administration 



Eligibility 

Missouri provided the following guidelines to be 
used in determining when a child’s significant 
cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

Parent signature was required 

•A Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

•A All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

•A No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

•A A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

•A Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
•A Student work samples only 

Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 

As part of day-to-day student instruction 
Separately from student’s daily work 

•A Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

•A Teacher decided content to be assessed 
•A Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Missouri 



Scoring and Reporting 
Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

One scorer 

• Who scored? 

Student’s classroom teacher, state or state- 
contracted scorer 

• How were scoring conflicts resolved? 

A third person adjudicated 
A third rater scored the assessment 

Y One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 
Ability to generalize across settings 

Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

Y None of the above 



Training 





Assessment 


Assessment 




administrators 


scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 




✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
Y' Scores 

Percentiles 

Interpretive guidance was developed for 
School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
■S Scores 

Y Standard/strand breakouts 
Indicator/benchmark breakouts 

Y Performance/achievement level descriptors 
Sample test items 



( ^ 

State department of education 

website 

http://dese.mo.gov/ 

V ) 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Missouri 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Grades 



3-5 

6-8 

9-12 



94 



93 



92 



0 20 



40 60 

Percent 



80 100 



Grades 



3-5 

6-8 



9-12 1 



Mathematics 



94 



93 



93 



0 20 



— i 1 — 

40 60 

Percent 



80 100 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 

Reading/Language Arts Mathematics 



Grades 


Grades 






186 


1 90 


3-5 


[TS 3-5 


[23 




□46 


□ 46 




1 73 


1 185 


6-8 


□TO 6-8 


M3 




□44 


□ 45 




□ 65 


181 


9-12 


1 6 9-12 


T7 




142 

1 1 1 1 1 


□41 

1 1 1 1 1 



0 20 40 60 80 100 0 20 40 60 80 100 

Percent Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.73 


0.76 


6-8 


0.62 


0.72 


9-12 


0.52 


0.60 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 



Montana 



Overview of the Alternate Criterion-Referenced Test (CRT-Alternate) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purpose of the 2006-07 Alternate Criterion- 
Referenced Test (CRT-Alternate) was to 

Evaluate programs 
Guide classroom instruction 

Measure student progress/performance toward state standards 
Y Assess student access to content standards/general curriculum 
Assess individual students’ strengths/weaknesses 
Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

Alternate Criterion-Referenced Test (CRT- 
Alternate) 

Alternate Assessment Approaches 

Rating scale/checklist 
Portfolio/body of evidence 

Y Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 
Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

Y More than 1 month to 2 months 
More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
4 of 5 standards 
Mathematics 

4 of 7 standards 

Who was involved in developing the alternate 
assessment? 

Y Assessment company 

Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 
Parents 
Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

Y A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Y Student’s classroom teacher 

Y School- or district-based educator 
State or state-contracted scorer 
Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Montana 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

State special education staff 
State assessment staff 
State instruction and curriculum staff 
•A Test vendor 
Outside experts 
•A Special education teachers 
•A General education teachers 
■A Content specialists 

School psychologists/counselors 
School/district/state administrators 
Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Bookmark or Item Mapping standard- 
setting methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Montana 
proficiency levels for 
alternate assessment 


Advanced 


Advanced 


Proficient 


Proficient 


Basic 


Nearing Proficiency 




Novice 



Alternate Achievement Descriptors 

Montana developed 

•A The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Proficient label descriptor in 
reading/language arts is as follows: 

• The student at the Proficient level, given 
limited prompting, demonstrates the ability 
to respond accurately in performing a wide 
variety of content specific performance 
indicators. 

Alternate Achievement Cut Scores 

Montana developed 
•A Unique cut scores for each grade 
Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Advanced and 
Proficient levels were proficient for purposes of 
the No Child Left Behind Act, whereas students 
who scored at the Nearing Proficiency and Novice 
levels were not proficient. 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

National Study on Alternate Assessments: State Profiles 



170 




Montana 



Technical Quality 



Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 




✓ 






State assessment staff 


✓ 






✓ 


State instruction and curriculum staff 










Test vendor 


✓ 


✓ 




✓ 


Outside experts 


✓ 


✓ 






Special education teachers 






✓ 


✓ 


General education teachers 






✓ 





Content specialists 
School psychologists/counselors 
School/district/state administrators 
Parents 



Validity 


Type of validity 


Types of evidence 


Scoring and reporting 
structures consistent with 
the subdomain 




•/ Relation to internal and 
external variables 


Item-to-item analysis 


V Purposes of the assessment 


Construct validity 
analysis 


Intended and unintended 
consequences 

Measurement of construct 
relevance 

Grade-level equating 





Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

General link to state content standards 

Y Grade or grade span 

Y Expanded benchmarks 
Alternate indicators or tasks 



✓ 



Reliability 



Type of reliability 


Types of evidence 


Variability across groups 




S Internal consistency of item 
responses 


Formal study or expert 
panel review 


S Interrater consistency in 
scoring 


Not available for 
examination 


Standard errors of 
measurement 




Other Types of Technical Quality 


Type of technical quality 


Types of evidence 





Alignment: 




✓ 


Initial 


Anecdotal or 
committee process 


✓ 


Ongoing/monitoring 


External and other 
alignment studies 


✓ 


Fairness 


Regularly scheduled 
bias review by experts 


✓ 


Implementation processes 


Training, 



administration manual/ 
guide, monitoring, post 
hoc data collection/ 



analysis 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

Montana Alternate Assessment for the 2006-07 School Year 



171 




Montana 



Eligibility and Administration 



Eligibility 

Montana provided the following guidelines to be 
used in determining when a child’s significant 
cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

Parent signature was required 

•A Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

•A All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

•A No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

•A A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

•A Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 

As part of day-to-day student instruction 
•A Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

Teacher decided content to be assessed 
•A Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Montana 



Scoring and Reporting 
Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

One scorer 

• Who scored? 

Student’s classroom teacher, school- or 
district-based educator 

• How were scoring conflicts resolved? 

A third person adjudicated 
A third rater scored the assessment 

Y One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 
Ability to generalize across settings 

Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

Y None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 


✓ 


✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
Y' Scores 

Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Performance/achievement level 
Y' Scores 

y' Standard/strand breakouts 
y' Indicator/benchmark breakouts 

Performance/achievement level descriptors 
Sample test items 



f \ 

State department of education 
website 

www.opi.state.mt.us/ 

V ) 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Montana 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Grades 



3-5 

6-8 

9-12 



93 



91 



86 



m 



— i — i — i — i — i 

20 40 60 80 100 

Percent 



Grades 



3-5 

6-8 

9-12 



Mathematics 



93 



92 



_8Z_ 



31 



20 40 60 80 100 

Percent 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts 




Percent 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.60 


0.60 


6-8 


0.56 


0.43 


9-12 


0.54 


0.44 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 



174 



National Study on Alternate Assessments: State Profiles 




Nebraska 



Overview of the School-based Teacher-led Assessment and Reporting System 
(STARS) Alternate Assessment (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 School-based Teacher- 
led Assessment and Reporting System (STARS) Alternate Assessment were to 

Evaluate programs 

Y Guide classroom instruction 

Y Measure student progress/performance toward state standards 
Assess student access to content standards/general curriculum 
Assess individual students’ strengths/weaknesses 

Y Document academic achievement 

Y Measure student progress toward IEP goals 



Alternate Assessment 

School-based Teacher-led Assessment and 
Reporting System (STARS) Alternate 
Assessment 

Alternate Assessment Approaches 

Rating scale/checklist 
Portfolio/body of evidence 

Y Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 

Y Science 

Y Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
4 of 13 standards 
Mathematics 

4 of 7 standards 

Who was involved in developing the alternate 
assessment? 

Assessment company 

Y Research company/university/independent 
researcher 

Y Technical assistance provider 

Y State personnel 

Y Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

Y A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Y Student’s classroom teacher 

Y School- or district-based educator 
State or state-contracted scorer 
Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Nebraska 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 
•A State instruction and curriculum staff 
Test vendor 
•A Outside experts 
•A Special education teachers 
•A General education teachers 
•A Content specialists 
•A School psychologists/counselors 
•A School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Bookmark or Item Mapping standard- 
setting methodology. 

Alternate Achievement Level Labels 



Alternate Achievement Descriptors 

Nebraska developed 

The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

•A Descriptors that were unique for each grade 
tested 

An example of the Proficient level descriptor in 
reading/language arts (grade 5) is as follows: 

• The student participates in structured group 
(two or more people) discussions. 

Alternate Achievement Cut Scores 

Nebraska developed 

Unique cut scores for each grade 
Unique cut scores for grade spans 
One set of cut scores for all students 

•A Other approaches (e.g., applying a rubric to 
determine proficiency level) 



NCLB-required 
proficiency levels 


Corresponding Nebraska 
proficiency levels for 
alternate assessment 


Advanced 


Advanced 


Proficient 


Proficient 


Basic 


Progressing 




Beginning 



Students who performed at the Advanced and 
Proficient levels were proficient for purposes of the 
No Child Left Behind Act, whereas students who 
scored at the Progressing and Beginning levels 
were not proficient. 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Nebraska 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


✓ 


✓ 


✓ 


✓ 


State assessment staff 


✓ 


✓ 


✓ 


✓ 


State instruction and curriculum staff 
Test vendor 


✓ 


✓ 


✓ 


✓ 


Outside experts 


✓ 


✓ 


✓ 


✓ 


Special education teachers 


✓ 


✓ 


✓ 


✓ 


General education teachers 


✓ 


✓ 


✓ 


✓ 


Content specialists 


✓ 


✓ 


✓ 


✓ 


School psychologists/counselors 


✓ 


✓ 


✓ 


✓ 


School/district/state administrators 


✓ 


✓ 


✓ 


✓ 


Parents 


✓ 


✓ 


✓ 


✓ 



Validity 


Type of validity 


Types of evidence 




Scoring and reporting 
structures consistent with 
the subdomain 






Relation to internal and 
external variables 




✓ 


Purposes of the assessment 


Anecdotal or 
committee process 




Intended and unintended 
consequences 




✓ 


Measurement of construct 
relevance 


Anecdotal or 
committee process 




Grade-level equating 





Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 
Grade or grade span 
Expanded benchmarks 
Alternate indicators or tasks 



Reliability 



Type of reliability Types of evidence 

Variability across groups 

Internal consistency of item 
responses 

■/ Interrater consistency in Statistical analysis 

scoring conducted as part of 

training 

Standard errors of 
measurement 



Other Types of Technical Quality 



Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Anecdotal or 
committee process 


✓ 


Ongoing/monitoring 


Internal alignment 
studies 


✓ 


Fairness 


Regularly scheduled 
bias review by experts, 
statistical analyses 


✓ 


Implementation processes 


Anecdotal or 
committee process 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Nebraska 



Eligibility and Administration 



Eligibility 

Nebraska provided the following guidelines to 
be used in determining when a child’s significant 
cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. 

Assessment Content Selection 

State determined 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

•A Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

•A No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

•A A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

State provided instructions on types and 
amounts of evidence to be collected 

•A Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
•A Student work samples only 

Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 
•A As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

•A Teacher decided content to be assessed 
•A Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Nebraska 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

One scorer 

• Who scored? 

Student’s classroom teacher, school- or 
district-based educator 

• How were scoring conflicts resolved? 

A third person adjudicated 
A third rater scored the assessment 

Y One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 

•Y Ability to generalize across settings 

Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

Y None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 







Individual Student Reports 

Individual student reports were given to 1 
Parents 

Schools and teachers 

Individual student results were expressed 
in terms of 1 

State’s achievement standards 

Scores 

Percentiles 

Interpretive guidance was developed for 1 
School-level administrators 
Teachers 
Parents 
Students 

Reports given to parents included the following 
information 1 

Performance/achievement level 
Scores 

Standard/strand breakouts 
Indicator/benchmark breakouts 
Performance/achievement level descriptors 
Sample test items 



f \ 

State department of education 
website 

www.nde.state.ne.us/ 

V ) 



1 Student reports and interpretive guidance were 
provided by local agencies. 

V Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Nebraska 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Grades 



3-5 

6-8 

9-12 



94 



92 



91 



0 20 



40 60 

Percent 



80 100 



Grades 



3-5 

6-8 

9-12 



Mathematics 



94 



92 



91 



0 20 



— i 1 — 

40 60 

Percent 



80 100 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 

Reading/Language Arts Mathematics 




Percent 




0 20 40 60 80 100 



Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.64 


0.68 


6-8 


0.75 


0.75 


9-12 


0.65 


0.64 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Data are unreliable for all students including students with lEPs in grades 3-12 who scored proficient or above on state 
accountability assessments in reading/language arts and mathematics. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, EDFacfs - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 



Nevada 



Overview of the Nevada Alternate Scales of Academic Achievement (NASAA) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Nevada Alternate Scales 
of Academic Achievement (NASAA) were to 

Evaluate programs 
Guide classroom instruction 

Y Measure student progress/performance toward state standards 

Y Assess student access to content standards/general curriculum 

Y Assess individual students’ strengths/weaknesses 
Document academic achievement 

Measure student progress toward IEP goals 



Alternate Assessment 

Nevada Alternate Scales of Academic 
Achievement (NASAA) 

Alternate Assessment Approaches 

Rating scale/checklist 
Portfolio/body of evidence 

Y Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 
Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ ✓ 
3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
3 of 12 standards 
Mathematics 

3 of 9 standards 

Who was involved in developing the alternate 
assessment? 

Y Assessment company 

Y Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 

Y Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

Y A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Y Student’s classroom teacher 
School- or district-based educator 

Y State or state-contracted scorer 
Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Nevada 



Alternate Achievement Standards 

Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 
•A State instruction and curriculum staff 
Test vendor 
•A Outside experts 
•A Special education teachers 
General education teachers 
•A Content specialists 

School psychologists/counselors 
•A School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Modified Angoff standard-setting 
methodology. 

Alternate Achievement Level Labels 



Alternate Achievement Descriptors 

Nevada developed 

•A The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Meets Standards descriptor in 
reading/language arts is as follows: 

• Performs basic components of skill with 
sufficient independence. 

Alternate Achievement Cut Scores 

Nevada developed 

Unique cut scores for each grade 
Unique cut scores for grade spans 
One set of cut scores for all students 

•A Other approaches (e.g., applying a rubric to 
determine proficiency level) 



NCLB-required 
proficiency levels 


Corresponding Nevada 
proficiency levels for 
alternate assessment 


Advanced 


Exceeds Standards 


Proficient 


Meets Standards 


Basic 


Below Standards 



Students who performed at the Exceeds Standards 
and Meets Standards levels were proficient for 
purposes of the No Child Left Behind Act, whereas 
students who scored at the Below Standards level 
were not proficient. 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Nevada 



Technical Quality 



Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 






✓ 


✓ 


State assessment staff 


✓ 






✓ 


State instruction and curriculum staff 




✓ 


✓ 


✓ 


Test vendor 










Outside experts 


V 


✓ 




✓ 


Special education teachers 






V 


✓ 


General education teachers 










Content specialists 








✓ 


School psychologists/counselors 










School/district/state administrators 






V 




Parents 






V 


✓ 



Validity Reliability 



Type of validity Types of evidence 

■/ Scoring and reporting Scoring and reporting 

structures consistent with documents 

the subdomain 

Relation to internal and 
external variables 

■/ Purposes of the assessment Survey 

✓ Intended and unintended Survey, public reports 

consequences 

✓ Measurement of construct Construct analyses 

relevance 

✓ Grade-level equating Anecdotal or 

committee process 



Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 

Y Grade or grade span 
•Y Expanded benchmarks 

Y Alternate indicators or tasks 



Type of reliability 


Types of evidence 




Variability across groups 






Internal consistency of item 
responses 




✓ 


Interrater consistency in 
scoring 


Statistical analysis 
conducted as part of 
actual scoring 




Standard errors of 
measurement 




Other Types of Technical Quality 


Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Formal study 


✓ 


Ongoing/monitoring 


Internal and external 
alignment studies 


✓ 


Fairness 


Bias review, not 
conducted regularly 


✓ 


Implementation processes 


Training, 
administration manual/ 
guide, post hoc data 
collection/analysis 



V Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Nevada 



Eligibility and Administration 



Eligibility 

Nevada provided the following guidelines to be 
used in determining when a child’s significant 
cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. 

Assessment Content Selection 

State determined 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

•A Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

•A An observer/monitor who was present 

A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

•A A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

•A Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 

As part of day-to-day student instruction 
Separately from student’s daily work 

•A Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

•A Teacher decided content to be assessed 
•A Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Nevada 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

Two scorers 

• Who scored? 

Student’s classroom teacher, state or state- 
contracted scorer 

• How were scoring conflicts resolved? 

A third person adjudicated 
Y A third rater scored the assessment 

One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 
Ability to generalize across settings 

•Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Y Appropriate human and technological 
supports 

None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 







Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

State’s achievement standards 

Y Scores 
Percentiles 

Interpretive guidance was developed for 1 
School-level administrators 
Teachers 
Parents 
Students 

Reports given to parents included the following 
Performance/achievement level 

Y Scores 

Standard/strand breakouts 
Indicator/benchmark breakouts 
Performance/achievement level descriptors 
Sample test items 



C A 

State department of education 
website 

www.doe.nv.gov/ 

V ) 



1 State did not provide interpretive guidance. 

V Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Nevada 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated 

Reading/Language Arts 




Percent 

□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



in state accountability assessments 

Mathematics 

Grades 




Percent 

□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts 




Percent 



Mathematics 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.67 


0.68 


6-8 


0.61 


0.64 


9-12 


0.53 


0.52 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 



New Hampshire 



Overview of the New Hampshire Alternate Assessment (NH-Alt) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 New Hampshire 
Alternate Assessment (NH-Alt) were to 

Y Evaluate programs 

Y Guide classroom instruction 

Y Measure student progress/performance toward state standards 

Y Assess student access to content standards/general curriculum 
Assess individual students’ strengths/weaknesses 

Y Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

New Hampshire Alternate Assessment (NH-Alt) 
Alternate Assessment Approaches 
Rating scale/checklist 

Y Portfolio/body of evidence 
Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 
Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓ ✓ ✓ ✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
2 of 8 standards 
Mathematics 

2 of 6 standards 

Who was involved in developing the alternate 
assessment? 

Y Assessment company 

Y Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 

Y Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Student’s classroom teacher 
School- or district-based educator 

Y State or state-contracted scorer 
Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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New Hampshire 



Alternate Achievement Standards 

Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 
•A State instruction and curriculum staff 
Test vendor 
•A Outside experts 
•A Special education teachers 
•A General education teachers 
•A Content specialists 

School psychologists/counselors 
•A School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Body of Work standard-setting 
methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding New Hampshire 
proficiency levels for 
alternate assessment 


Advanced 


Proficient with Distinction 


Proficient 


Proficient 


Basic 


Partially Proficient 




Substantially Below Proficient 



Students who performed at the Proficient with 
Distinction and Proficient levels were proficient for 
purposes of the No Child Left Behind Act, whereas 
students who scored at the Partially Proficient 
and Substantially Below Proficient levels were not 
proficient. 



Alternate Achievement Descriptors 

New Hampshire developed 

•A The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Proficient descriptor in reading/ 

language arts is as follows: 

• Student is successfully demonstrating 

moderate progress that is consistent with the 
intended goal(s) in targeted reading skill(s). 
Student has access to and is using a variety 
of modified reading materials that are linked 
to general education reading curriculum 
activities. Opportunities to practice reading 
skills are offered in varied settings, or 
consistently within a general education or 
other natural setting. Opportunities for self- 
determination and interaction with typical 
peers are consistent. Instructional supports, 
team supports, and/or task structure are 
adequate for this student to access modified 
grade-linked reading materials and/or 
activities in a manner that promotes skill 
progress, generalization of performance, 
and self-determination. Remaining areas of 
weakness can be addressed by the existing 
team. 

Alternate Achievement Cut Scores 

New Hampshire developed 

Unique cut scores for each grade 
Unique cut scores for grade spans 

•A One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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New Hampshire 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


✓ 




✓ 


✓ 


State assessment staff 


✓ 


✓ 


✓ 


✓ 


State instruction and curriculum staff 
Test vendor 


✓ 


✓ 


✓ 


✓ 


Outside experts 


✓ 




✓ 


✓ 


Special education teachers 


✓ 




✓ 


✓ 


General education teachers 


✓ 




✓ 


✓ 


Content specialists 

School psychologists/counselors 

School/district/state administrators 


✓ 




✓ 


✓ 


Parents 


✓ 




✓ 


✓ 



Validity 



Type of validity Types of evidence 





Scoring and reporting 
structures consistent with 
the subdomain 






Relation to internal and 
external variables 




✓ 


Purposes of the assessment 


Anecdotal or 
committee process 


✓ 


Intended and unintended 
consequences 


Survey 


✓ 


Measurement of construct 
relevance 


Construct analyses 




Grade-level equating 





Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 
Grade or grade span 
Expanded benchmarks 
Alternate indicators or tasks 



Reliability 



Type of reliability 


Types of evidence 


✓ 


Variability across groups 


NCLB group statistical 
analyses 




Internal consistency of item 
responses 




✓ 


Interrater consistency in 
scoring 


Statistical analysis 
conducted as part of 
actual scoring 




Standard errors of 
measurement 




Other Types of Technical Quality 


Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Formal study 




Ongoing/monitoring 




✓ 


Fairness 


Bias review, not 
conducted regularly 


✓ 


Implementation processes 


Training, 
administration manual/ 
guide 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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New Hampshire 



Eligibility and Administration 



Eligibility 

New Hampshire provided the following guidelines 
to be used in determining when a child’s 
significant cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

•A Parent signature was required 

Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

•A Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
•A Student work samples only 

Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 
•A As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

•A Teacher decided content to be assessed 
•A Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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New Hampshire 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

Two scorers 

• Who scored? 

State or state-contracted scorer 

• How were scoring conflicts resolved? 

A third person adjudicated 
Y A third rater scored the assessment 

One person scored, or scores were 
combined 

Elements of student performance used in scoring 
Accuracy of student response 

Y Ability to generalize across settings 

Y Amount of independence 

Y Amount of progress 

Environmental elements used in scoring 

Y Instruction in multiple settings 

Y Opportunities to plan, monitor, and evaluate 
work 

Y Work with nondisabled peers 

Y Appropriate human and technological 
supports 

None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 




✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
Y' Scores 

Y Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
Y' Scores 

Standard/strand breakouts 
Indicator/benchmark breakouts 
y' Performance/achievement level descriptors 
y' Sample test items 



( ^ 

State department of education 

website 

www.ed.state.nh.us/assessment 

V J 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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New Hampshire 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Mathematics 



Grades 




□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Grades 




□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts 




Percent 



Mathematics 



Grades 



3-5 



58 

□67 



6-8 



9-12 



M 

M 

M 



153 

□62 



T 



1 



0 20 40 60 80 100 



Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ Students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.10 


0.62 


6-8 


0.38 


0.46 


9-12 


M 


M 



M Missing. Information not provided by state. 

1 Of all students who received valid scores on the specific types of accountability assessment. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 




New Jersey 



Overview of the Alternate Proficiency Assessment (APA) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Alternate Proficiency 
Assessment (APA) were to 

Y Evaluate programs 

Guide classroom instruction 

Y Measure student progress/performance toward state standards 

Y Assess student access to content standards/general curriculum 
Assess individual students’ strengths/weaknesses 
Document academic achievement 

Measure student progress toward IEP goals 



Alternate Assessment 

Alternate Proficiency Assessment (APA) 
Alternate Assessment Approaches 
Rating scale/checklist 

Y Portfolio/body of evidence 
Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 

Y Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
2 of 5 standards 
Mathematics 

2 of 5 standards 

Who was involved in developing the alternate 
assessment? 

Y Assessment company 

Y Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 
Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Student’s classroom teacher 
School- or district-based educator 

Y State or state-contracted scorer 
Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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New Jersey 



Alternate Achievement Standards 

Who was involved in creating alternate 
achievement standards? 

•A State special education staff 

•A State assessment staff 

•A State instruction and curriculum staff 

•A Test vendor 

•A Outside experts 

•A Special education teachers 

•A General education teachers 

•A Content specialists 

•A School psychologists/counselors 

•A School/district/state administrators 

•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Body of Work standard-setting 
methodology. 

Alternate Achievement Level Labels 



Alternate Achievement Descriptors 

New Jersey developed 

•A The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Proficient descriptor in reading/ 
language arts is as follows: 

• Proficient indicates that the portfolio met the 
state level of proficiency in the content area. 

Alternate Achievement Cut Scores 

New Jersey developed 

Unique cut scores for each grade 
Unique cut scores for grade spans 
•A One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



NCLB-required 
proficiency levels 


Corresponding New Jersey 
proficiency levels for 
alternate assessment 


Advanced 


Advanced Proficient 


Proficient 


Proficient 


Basic 


Partially Proficient 



Students who performed at the Advanced Proficient 
and Proficient levels were proficient for purposes 
of the No Child Left Behind Act, whereas students 
who scored at the Partially Proficient level were not 
proficient. 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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New Jersey 



Technical Quality 



Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


✓ 


✓ 


✓ 


✓ 


State assessment staff 


✓ 


✓ 


✓ 


✓ 


State instruction and curriculum staff 


✓ 


✓ 


✓ 


✓ 


Test vendor 


✓ 


✓ 


✓ 


✓ 


Outside experts 


✓ 


✓ 


✓ 


✓ 


Special education teachers 
General education teachers 


✓ 


✓ 


✓ 


✓ 


Content specialists 


✓ 


✓ 


✓ 


✓ 


School psychologists/counselors 
School/district/state administrators 


✓ 




✓ 


✓ 


Parents 


✓ 


✓ 


✓ 


✓ 



Validity Reliability 



Type of validity Types of evidence 



✓ 


Scoring and reporting 
structures consistent with 
the subdomain 


Not available for 
examination 


✓ 


Relation to internal and 
external variables 


Item-to-item analysis 


✓ 


Purposes of the assessment 


Anecdotal or 
committee process 


✓ 


Intended and unintended 
consequences 


Not available for 
examination 


✓ 


Measurement of construct 
relevance 

Grade-level equating 


Anecdotal or 
committee process 



Extended Content Standards 

The state did not develop extended content 
standards for students who were eligible for the 
alternate assessment. 



Type of reliability Types of evidence 



■/ Variability across groups 


Not available for 
examination 


Internal consistency of item 
responses 




S Interrater consistency in 
scoring 


Not available for 
examination 


Standard errors of 
measurement 





Other Types of Technical Quality 



Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Anecdotal or 






committee process 


✓ 


Ongoing/monitoring 


Internal alignment 
studies 




Fairness 




✓ 


Implementation processes 


Training, 



administration manual/ 
guide, post hoc data 
collection/analysis 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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New Jersey 



Eligibility and Administration 



Eligibility 

New Jersey provided the following guidelines 
to be used in determining when a child’s 
significant cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

Parent signature was required 

•A Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

•A Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
•A Student work samples only 

Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 
•A As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

•A Teacher decided content to be assessed 
•A Teacher selected materials 

Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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New Jersey 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

Two scorers 

• Who scored? 

State or state-contracted scorer 

• How were scoring conflicts resolved? 

A third person adjudicated 
Y A third rater scored the assessment 

One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 
Ability to generalize across settings 

Y Amount of independence 

Y Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Y Appropriate human and technological 
supports 

None of the above 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

State’s achievement standards 
J' Scores 
Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
■S Scores 

Y Standard/strand breakouts 
Indicator/benchmark breakouts 

Y Performance/achievement level descriptors 
Sample test items 



Training 





Assessment 


Assessment 


r 


\ 


Non-face-to-face 

training 


administrators 

✓ 


scorers 


State department of education 
website 




Face-to-face 

training 


✓ 


✓ 


www. nj . g ov/ed u cati on/ 

V 


) 



Mandatory 


✓ 


training and/or 




certification 





V Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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New Jersey 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Grades 



3-5 

6-8 

9-12 



93 



93 



91 



0 20 



40 60 

Percent 



80 100 



Grades 



3-5 

6-8 

9-12 



Mathematics 



93 



93 



m 



91 



0 20 



— i 1 — 

40 60 

Percent 



80 100 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts 



Grades 






190 


3-5 


157 




M 




1 89 


6-8 


f39 




M 




1 88 


9-12 


H[48 




M 

1 1 1 1 1 



0 20 40 60 80 100 

Percent 



Mathematics 



Grades 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.85 


0.82 


6-8 


0.81 


0.80 


9-12 


0.83 


0.77 



M Missing. Information not provided by state. 

1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 




New Mexico 



Overview of the New Mexico Alternate Performance Assessment (NMAPA) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 New Mexico Alternate 
Performance Assessment (NMAPA) were to 

Evaluate programs 
•Y Guide classroom instruction 

•Y Measure student progress/performance toward state standards 

Y Assess student access to content standards/general curriculum 

Y Assess individual students’ strengths/weaknesses 

Y Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

New Mexico Alternate Performance Assessment 
(NMAPA) 

Alternate Assessment Approaches 

Rating scale/checklist 
Portfolio/body of evidence 

Y Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 

Y Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓✓✓✓✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

Y More than 1 month to 2 months 
More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 

3 of 3 standards 
Mathematics 

4 standards 1 

Who was involved in developing the alternate 
assessment? 

Assessment company 

Y Research company/university/independent 
researcher 

Y Technical assistance provider 

Y State personnel 
Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

Y A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Y Student’s classroom teacher 

Y School- or district-based educator 
State or state-contracted scorer 
Machine scored 
Paraprofessional 

1 The number of general content standards varied by grade 
level. 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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New Mexico 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 

•A State assessment staff 

•A State instruction and curriculum staff 

•A Test vendor 

•A Outside experts 

•A Special education teachers 

•A General education teachers 

■A Content specialists 

School psychologists/counselors 
School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Item-Descriptor Matching standard- 
setting methodology. 



Alternate Achievement Descriptors 

New Mexico developed 

The same descriptors that applied to all 
grades tested 

•A The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Proficient descriptor in reading/ 
language arts (grades 3 and 4) is as follows: 

• A student scoring at the Proficient level 
(grade 3-4) should be able to: read and 
comprehend relevant academic vocabulary; 
follow simple directions in familiar contexts; 
identify and classify a variety of media 
formats (e.g., books, newspapers, videos); 
identify the main character in a text; 
demonstrate an understanding that graphic 
information (pictures, print, tactile symbols) 
has meaning; sequence events in a story. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding New Mexico 
proficiency levels for 
alternate assessment 


Advanced 


Advanced 


Proficient 


Proficient 


Basic 


Nearing Proficiency 




Beginning Steps 



Alternate Achievement Cut Scores 

New Mexico developed 

Unique cut scores for each grade 
•A Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Advanced and 
Proficient levels were proficient for purposes of the 
No Child Left Behind Act, whereas students who 
scored at the Nearing Proficiency and Beginning 
Steps levels were not proficient. 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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New Mexico 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 










State assessment staff 


✓ 


✓ 


✓ 


✓ 


State instruction and curriculum staff 








✓ 


Test vendor 


✓ 


✓ 


✓ 


✓ 


Outside experts 


✓ 


✓ 


✓ 


✓ 


Special education teachers 


✓ 


✓ 


✓ 


✓ 


General education teachers 


✓ 


✓ 


✓ 


✓ 


Content specialists 


✓ 


✓ 


✓ 


✓ 


School psychologists/counselors 










School/district/state administrators 


✓ 


✓ 


✓ 


✓ 


Parents 


✓ 


✓ 


✓ 


✓ 



Validity 



Type of validity 


Types of evidence 




Scoring and reporting 
structures consistent with 
the subdomain 




✓ 


Relation to internal and 
external variables 


Item-to-item analysis 


✓ 


Purposes of the assessment 


State monitoring/ 
program review 


✓ 


Intended and unintended 


Post hoc data 




consequences 


collection/analysis 


✓ 


Measurement of construct 
relevance 


Construct analyses 


✓ 


Grade-level equating 


Formal study or expert 
panel review 



Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 

Y Grade or grade span 

Y Expanded benchmarks 
Alternate indicators or tasks 



Reliability 



Type of reliability 


Types of evidence 


✓ 


Variability across groups 


NCLB group 
statistical analyses 


✓ 


Internal consistency of item 
responses 


Formal study or expert 
panel review 


✓ 


Interrater consistency in 
scoring 


Statistical analysis 
conducted as part of 
actual scoring 


✓ 


Standard errors of 
measurement 


Statistical analysis 


Other Types of Technical Quality 


Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Formal study 


✓ 


Ongoing/monitoring 


Internal and external 
alignment studies 


✓ 


Fairness 


Regularly scheduled 
bias review by experts 


✓ 


Implementation processes 


Training 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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New Mexico 



Eligibility and Administration 



Eligibility 

New Mexico provided the following guidelines 
to be used in determining when a child’s 
significant cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. 

Assessment Content Selection 

State determined 

•A All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

•A A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

•A State required standardized tasks/test items/ 
rating scales 

State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

Combination of work samples and other 
evidence 

•A No student work samples 

Assessment was administered 

As part of day-to-day student instruction 
•A Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

Teacher decided content to be assessed 
Teacher selected materials 

Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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New Mexico 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

Two scorers 

• Who scored? 

Student’s classroom teacher, school- or 
district-based educator 

• How were scoring conflicts resolved 2 1 

A third person adjudicated 
A third rater scored the assessment 

One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 
Ability to generalize across settings 
Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

Y None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 


✓ 


✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
Y' Scores 

Y Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
Y' Scores 

Standard/strand breakouts 
Indicator/benchmark breakouts 
y' Performance/achievement level descriptors 
Sample test items 



( ^ 

State department of education 

website 

http://ped.state.nm.us 

V ) 



1 The score posted by the original scorer prevailed if 
there was a scoring conflict. 

Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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New Mexico 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Grades 



3-5 

6-8 

9-12 



94 



94 



93 



0 20 



40 60 

Percent 



80 100 



Grades 



3-5 

6-8 

9-12 



Mathematics 



94 



94 



93 



0 20 



— i 1 — 

40 60 

Percent 



80 100 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts 



Grades 



3-5 



6-8 



9-12 



HL 



155 



U54 



TFT 



□ 67 



□46 



~ ITT 



] 59 



19 



— i 1 1 1 1 

0 20 40 60 80 100 
Percent 



Grades 



Mathematics 

] 35 




9-12 



T 1 1 1 1 

0 20 40 60 80 100 
Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.47 


0.30 


6-8 


0.65 


0.48 


9-12 


0.48 


0.40 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 



New York 



Overview of the New York State Alternate Assessment (NYSAA) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purpose of the 2006-07 New York State Alternate 
Assessment (NYSAA) was to 

Evaluate programs 
Guide classroom instruction 

Y Measure student progress/performance toward state standards 
Assess student access to content standards/general curriculum 
Assess individual students’ strengths/weaknesses 
Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

New York State Alternate Assessment (NYSAA) 
Alternate Assessment Approaches 
Rating scale/checklist 

Y Portfolio/body of evidence 
Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 

Y Science 

Y Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
Varied by grade level 1 
Mathematics 

1 of 5 standards 

Who was involved in developing the alternate 
assessment? 

Y Assessment company 

Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 
Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

Y A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Student’s classroom teacher 
School- or district-based educator 

Y State or state-contracted scorer 
Machine scored 
Paraprofessional 



1 The number of general content standards was 4. 

V Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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New York 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 

•A State assessment staff 

•A State instruction and curriculum staff 

•A Test vendor 

•A Outside experts 

•A Special education teachers 

•A General education teachers 

•A Content specialists 

School psychologists/counselors 
•A School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Body of Work standard-setting 
methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding New York 
proficiency levels for 
alternate assessment 


Advanced 


Meeting with Distinction 


Proficient 


Meeting 


Basic 


Partially Meeting 




Not Meeting 



Students who performed at the Meeting with 
Distinction and Meeting levels were proficient for 
purposes of the No Child Left Behind Act, whereas 
students who scored at the Partially Meeting and 
Not Meeting levels were not proficient. 



Alternate Achievement Descriptors 

New York developed 

The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

•A Descriptors that were unique for each grade 
tested 

An example of the Meeting Learning Standards 
descriptor in reading/language arts (grade 3) is as 
follows: 

• Student’s performance is often accurate. 
Student often demonstrates an 
understanding of AGLIs. Student actively 
engages in tasks with occasional support 
(i.e., cues and/or prompts beyond scope of 
IEP accommodations). Student often attends 
to and/or reads to build comprehension and/ 
or make simple predictions; and/or, Student 
often engages in the selection of books; 
and/or Student often engages in discussion 
with classmates about texts (i.e., listens 
to others’ thoughts and opinions, answers 
“wh” questions, etc.); and/or Student often 
identifies words, conventions, purposes, tone 
and/or feelings. 

Alternate Achievement Cut Scores 

New York developed 

•A Unique cut scores for each grade 
Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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New York 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


✓ 


✓ 


✓ 


✓ 


State assessment staff 


✓ 




✓ 


✓ 


State instruction and curriculum staff 






✓ 




Test vendor 


✓ 


✓ 






Outside experts 


✓ 


✓ 


✓ 




Special education teachers 


✓ 




✓ 


✓ 


General education teachers 


✓ 




✓ 


✓ 


Content specialists 


✓ 




✓ 


✓ 


School psychologists/counselors 










School/district/state administrators 


✓ 




✓ 




Parents 











Validity 



Type of validity 

Scoring and reporting 
structures consistent with 
the subdomain 
Relation to internal and 
external variables 

■/ Purposes of the assessment 

■/ Intended and unintended 
consequences 

■/ Measurement of construct 
relevance 

Grade-level equating 



Types of evidence 



Survey 

Survey, post hoc data 
collection/analysis 

Not available for 
examination 



Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 

Y Grade or grade span 
•Y Expanded benchmarks 

•Y Alternate indicators or tasks 



Reliability 



Type of reliability 


Types of evidence 




Variability across groups 




✓ 


Internal consistency of item 
responses 


Formal study or expert 
panel review 


✓ 


Interrater consistency in 
scoring 


Statistical analysis 
conducted as part of 
training and actual 
scoring 




Standard errors of 
measurement 




Other Types of Technical Quality 


Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Formal study 


✓ 


Ongoing/monitoring 


External alignment 
study 


✓ 


Fairness 


Regularly scheduled 
bias review by experts 


✓ 


Implementation processes 


Training, 
administration manual/ 
guide, monitoring, post 
hoc data collection/ 
analysis 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

New York Alternate Assessment for the 2006-07 School Year 



207 




New York 



Eligibility and Administration 



Eligibility 

New York provided the following guidelines to 
be used in determining when a child’s significant 
cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

Y' Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

Required modified curriculum 
Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. 

Assessment Content Selection 

State determined 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Y' Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

Y' A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
Y' The student’s special education teacher 

Y' A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

Y' State provided instructions on types and 
amounts of evidence to be collected 

Y' Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Y' Student work samples only 

Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 
Y' As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

Y' Teacher decided content to be assessed 
Y' Teacher selected materials 

Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

Y' Teacher interpreted/recorded student 
responses or scores 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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New York 



Scoring and Reporting 
Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 1 
One scorer 

• Who scored? 

State or state-contracted scorer 

• How were scoring conflicts resolved? 

Y A third person adjudicated 

A third rater scored the assessment 

One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 
Ability to generalize across settings 

Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

Y None of the above 



Training 





Assessment 


Assessment 




administrators 


scorers 


Non-face-to-face 

training 


✓ 




Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 




✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
Scores 

Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 
Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
Scores 

Y Standard/strand breakouts 
Indicator/benchmark breakouts 

Y Performance/achievement level descriptors 
Sample test items 



( \ 

State department of education 

website 

www.nysed.gov/ 

V ) 



1 Portion of sample double scored. 

S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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New York 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Grades 



3-5 

6-8 

9-12 



92 



90 



79 



T151 



— i 1 1 1 1 

20 40 60 80 100 

Percent 



Grades 



3-5 

6-8 

9-12 



Mathematics 



92 



90 



83 



132 ] 



20 40 60 80 100 

Percent 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Grades 



3-5 



6-8 



9-12 



Reading/Language Arts 

188 



~mr 



368 



3ZLZ1 



190 



360 



1ST 



□ 90 
388 



— i 1 1 1 1 

0 20 40 60 80 100 
Percent 



Mathematics 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.84 


0.87 


6-8 


0.93 


0.91 


9-12 


0.53 


0.50 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 




North Carolina 



Overview of the North Carolina EXTENDI (NCEXTEND1) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 North Carolina 
EXTENDI (NCEXTEND1) were to 

Evaluate programs 
Guide classroom instruction 

X Measure student progress/performance toward state standards 
Assess student access to content standards/general curriculum 
X Assess individual students’ strengths/weaknesses 
Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

North Carolina EXTENDI (NCEXTEND1) 
Alternate Assessment Approaches 
Rating scale/checklist 
Portfolio/body of evidence 
X Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 
X Reading/language arts 
X Mathematics 
X Science 

Social studies 
Functional skills 
Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 
More than 2 weeks to 1 month 
X More than 1 month to 2 months 
More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
Varied by grade level 
Mathematics 

5 of 5 standards 

Who was involved in developing the alternate 
assessment? 

Assessment company 

X Research company/university/independent 
researcher 

Technical assistance provider 
X State personnel 
Parents 

X Stakeholders 

Who administered the alternate assessment? 

X The student’s special education teacher 

X A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

X Student’s classroom teacher 
X School- or district-based educator 
State or state-contracted scorer 
Machine scored 
Paraprofessional 



X Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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North Carolina 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 
•A State instruction and curriculum staff 
Test vendor 
•A Outside experts 
•A Special education teachers 
General education teachers 
Content specialists 
School psychologists/counselors 
•A School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Reasoned Judgment and Contrasting 
Groups standard-setting methodologies. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding North Carolina 
proficiency levels for 
alternate assessment 


Advanced 


Achievement Level IV 


Proficient 


Achievement Level III 


Basic 


Achievement Level II 




Achievement Level 1 



Alternate Achievement Descriptors 

North Carolina developed 

•A The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Achievement Level III descriptor 
in reading/language arts is as follows: 

• Students performing at this level often 
demonstrate mastery of the NCSCS 
Extended Content Standards in English 
language arts. 

Alternate Achievement Cut Scores 

North Carolina developed 

Unique cut scores for each grade 
Unique cut scores for grade spans 
•A One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Achievement 
Level IV and Achievement Level III were proficient 
for purposes of the No Child Left Behind Act, 
whereas students who scored at the Achievement 
Level II and Achievement Level I were not 
proficient. 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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North Carolina 



Technical Quality 



Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


✓ 


✓ 


✓ 


✓ 


State assessment staff 


✓ 


✓ 


✓ 


✓ 


State instruction and curriculum staff 
Test vendor 


✓ 


✓ 


✓ 


✓ 


Outside experts 


✓ 




✓ 


✓ 


Special education teachers 


✓ 




✓ 


✓ 


General education teachers 
Content specialists 
School psychologists/counselors 
School/district/state administrators 
Parents 


✓ 


✓ 




✓ 

✓ 



Validity Reliability 



Type of validity 


Types of evidence 


Type of reliability 


Types of evidence 


Scoring and reporting 




✓ 


Variability across groups 


Not available for 


structures consistent with 








examination 


the subdomain 




✓ 


Internal consistency of item 


Not available for 


✓ Relation to internal and 


Not available for 




responses 


examination 


external variables 


examination 


✓ 


Interrater consistency in 


Training documents or 


V Purposes of the assessment 


Not available for 




scoring 


anecdotal 




examination 




Standard errors of 




V Intended and unintended 


Not available for 




measurement 




consequences 


examination 








S Measurement of construct 


Not available for 








relevance 


examination 


Other Types of Technical Quality 


■/ Grade-level equating 


Not available for 
















examination 












Type of technical quality 


Types of evidence 


Extended Content Standards 






Alignment: 








✓ 


Initial 


Formal study 


The state developed extended content standards 


✓ 


Ongoing/monitoring 


Internal, external, 


for students who were eligible for the alternate 




and other alignment 


assessment. 








studies 


Y General link to state content standards 




Fairness 


Bias review, not 


Y Grade or grade span 








conducted regularly 




✓ 


Implementation processes 


Not available for 


Expanded benchmarks 








examination 



Alternate indicators or tasks 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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North Carolina 



Eligibility and Administration 



Eligibility 

North Carolina provided the following guidelines 
to be used in determining when a child’s 
significant cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. 

Assessment Content Selection 

State determined 

•A All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

•A An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

•A A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

•A State required standardized tasks/test items/ 
rating scales 

State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

Combination of work samples and other 
evidence 

•A No student work samples 

Assessment was administered 

As part of day-to-day student instruction 
•A Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

Teacher decided content to be assessed 
Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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North Carolina 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

Two scorers 

• Who scored? 

Student’s classroom teacher, school- or 
district-based educator 

• How were scoring conflicts resolved? 

A third person adjudicated 
A third rater scored the assessment 

Y One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 
Ability to generalize across settings 
Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

Y None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 


✓ 


v' 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
Y' Scores 

Percentiles 

Interpretive guidance was developed for 1 
School-level administrators 
Teachers 
Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
Y' Scores 

Standard/strand breakouts 
Indicator/benchmark breakouts 
y' Performance/achievement level descriptors 
Sample test items 



C A 

State department of education 
website 

www.ncpublicschools.org/ 

V ) 



1 State did not provide interpretive guidance. 

Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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North Carolina 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Mathematics 




Percent 




□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 1 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 1 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 2 



Reading/Language Arts 




Percent 



Mathematics 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


3.42 


3.03 


6-8 


3.05 


2.86 


9-12 


2.59 


0.87 



1 For this state this included an alternate assessment based on grade-level achievement standards. 

2 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. Data are unreliable for students 
with an IEP taking an alternate assessment based on alternate achievement standards and regular assessment with or without 
accommodations in grades 3-12 who scored proficient or above on state accountability assessments in reading/language arts 
and mathematics. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 



216 



National Study on Alternate Assessments: State Profiles 



North Dakota 



Overview of the North Dakota Alternate Assessment (NDAA) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 North Dakota Alternate 
Assessment (NDAA) were to 

Evaluate programs 

Y Guide classroom instruction 

Y Measure student progress/performance toward state standards 

Y Assess student access to content standards/general curriculum 

Y Assess individual students’ strengths/weaknesses 

Y Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

North Dakota Alternate Assessment (NDAA) 
Alternate Assessment Approaches 

Y Rating scale/checklist 
Portfolio/body of evidence 

Y Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 

Y Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

Y More than 1 month to 2 months 
More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
3 of 6 standards 
Mathematics 

5 of 5 standards 

Who was involved in developing the alternate 
assessment? 

Assessment company 

Y Research company/university/independent 
researcher 

Y Technical assistance provider 

Y State personnel 

Y Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Y Student’s classroom teacher 
School- or district-based educator 
State or state-contracted scorer 
Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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North Dakota 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 
•A State instruction and curriculum staff 
Test vendor 
•A Outside experts 
•A Special education teachers 
•A General education teachers 
•A Content specialists 

School psychologists/counselors 
•A School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Modified Angoff standard-setting 
methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding North Dakota 
proficiency levels for 
alternate assessment 


Advanced 


Advanced 


Proficient 


Proficient 


Basic 


Partially Proficient 




Novice 



Alternate Achievement Descriptors 

North Dakota developed 

The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

•A Descriptors that were unique for each grade 
tested 

An example of the Proficient descriptor in reading/ 
language arts (grade 3) is as follows: 

• Student recognizes nonfiction text; 

recognizes and recalls symbols; interacts 
with reading; identifies main ideas. Student 
selects, plans, and evaluates performance in 
multiple settings. 

Alternate Achievement Cut Scores 

North Dakota developed 

Unique cut scores for each grade 
Unique cut scores for grade spans 
•A One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Advanced and 
Proficient levels were proficient for purposes of the 
No Child Left Behind Act, whereas students who 
scored at the Partially Proficient and Novice levels 
were not proficient. 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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North Dakota 



Technical Quality 



Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 










State assessment staff 


✓ 


✓ 




✓ 


State instruction and curriculum staff 








✓ 


Test vendor 










Outside experts 


V 


✓ 


✓ 


✓ 


Special education teachers 






✓ 


✓ 


General education teachers 






✓ 


✓ 


Content specialists 






✓ 


✓ 


School psychologists/counselors 










School/district/state administrators 


V 


✓ 


✓ 


✓ 


Parents 


V 




✓ 


✓ 



Validity 



Type of validity 


Types of evidence 




Scoring and reporting 
structures consistent with 
the subdomain 




✓ 


Relation to internal and 
external variables 


Committee process or 
internal review 


✓ 


Purposes of the assessment 


Anecdotal or 
committee process 


✓ 


Intended and unintended 
consequences 


Anecdotal or 
committee process 


✓ 


Measurement of construct 
relevance 


Anecdotal or 
committee process 




Grade-level equating 





Extended Content Standards 

The state did not develop extended content 
standards for students who were eligible for the 
alternate assessment. 



Reliability 


Type of reliability 


Types of evidence 


■/ Variability across groups 


NCLB group statistical 
analyses 


Internal consistency of item 
responses 

Interrater consistency in 
scoring 




✓ Standard errors of 
measurement 


Statistical analysis 



Other Types of Technical Quality 



Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Anecdotal or 
committee process 


✓ 


Ongoing/monitoring 


Internal alignment 
studies 




Fairness 


Bias review, not 
conducted regularly 


✓ 


Implementation processes 


Training, post hoc data 
collection/analysis 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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North Dakota 



Eligibility and Administration 



Eligibility 

North Dakota provided the following guidelines 
to be used in determining when a child’s 
significant cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. 

Assessment Content Selection 

State determined 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

•A Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

•A No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

•A State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

Combination of work samples and other 
evidence 

•A No student work samples 

Assessment was administered 

As part of day-to-day student instruction 
•A Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

•A Teacher decided content to be assessed 
•A Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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North Dakota 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

One scorer 

• Who scored? 

Student’s classroom teacher 

• How were scoring conflicts resolved? 

A third person adjudicated 
A third rater scored the assessment 

Y One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 

Y Ability to generalize across settings 

Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 

Y Instruction in multiple settings 

Y Opportunities to plan, monitor, and evaluate 
work 

Y Work with nondisabled peers 

•Y Appropriate human and technological 
supports 

None of the above 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
Scores 

Percentiles 

Interpretive guidance was developed for 
School-level administrators 
Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
Scores 

Standard/strand breakouts 

Y Indicator/benchmark breakouts 

Y Performance/achievement level descriptors 
Sample test items 



Training 





Assessment 


Assessment 


r 


\ 




administrators 


scorers 


State department of education 




Non-face-to-face 


✓ 


✓ 


website 




training 

Face-to-face 

training 






www. dpi. state . nd . us/ 

V 


) 










Mandatory 
training and/or 
certification 











V Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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North Dakota 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Mathematics 



Grades Grades 





Percent 



Percent 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts 

Grades 


Mathematics 


190 


189 


T62 3-5 


fBS— ^ 


179 


Z]82 






1 90 


192 


153 6-8 


[44 


178 


175 






183 


192 


[35 9-12 


113 


172 


157 


1 1 1 1 1 


1 1 1 1 1 



0 20 40 60 80 100 0 20 40 60 80 100 

Percent Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


2.13 


1.94 


6-8 


2.39 


2.53 


9-12 


1.83 


2.37 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 




Ohio 



Overview of the Alternate Assessment for Students with Disabilities (AASWD) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Alternate Assessment for 
Students with Disabilities (AASWD) were to 

Evaluate programs 
Guide classroom instruction 

Y Measure student progress/performance toward state standards 
Assess student access to content standards/general curriculum 
Assess individual students’ strengths/weaknesses 

Y Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

Alternate Assessment for Students 
with Disabilities (AASWD) 

Alternate Assessment Approaches 

Rating scale/checklist 

Y Portfolio/body of evidence 
Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 

Y Science 

Y Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
8 of 8 standards 
Mathematics 

6 of 6 standards 

Who was involved in developing the alternate 
assessment? 

Y Assessment company 

Y Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 
Parents 
Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

Y A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Student’s classroom teacher 
School- or district-based educator 

Y State or state-contracted scorer 
Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Ohio 



Alternate Achievement Standards 

Who was involved in creating alternate 
achievement standards? 

State special education staff 
State assessment staff 
State instruction and curriculum staff 
•A Test vendor 
•A Outside experts 
•A Special education teachers 
General education teachers 
Content specialists 
School psychologists/counselors 
•A School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Body of Work standard-setting 
methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Ohio 
proficiency levels for 
alternate assessment 


Advanced 


Advanced 




Accelerated 


Proficient 


Proficient 


Basic 


Basic 




Limited 



Alternate Achievement Descriptors 

Ohio developed 

The same descriptors that applied to all 
grades tested 

•A The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Proficient descriptor in reading/ 

language arts (grades 3 to 5) is as follows: 

• The student often demonstrates reading 
skills, such as, identifying the beginning, 
middle and/or ending sounds of words, using 
picture symbols to express ideas from a 
literary text and using symbols to express 
ideas, needs and/or wants, which align with 
and access Grades 3-5 reading content from 
the Ohio Academic Content Standards. The 
student performs the skill accurately some 
of the time, needs some supervision in the 
classroom and may not be ready to apply 
reading skills in real world situations. 

Alternate Achievement Cut Scores 

Ohio developed 

•A Unique cut scores for each grade 1 

•A Unique cut scores for grade spans 2 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Advanced, 
Accelerated, and Proficient levels were proficient 
for purposes of the No Child Left Behind Act, 
whereas students who scored at the Basic and 
Limited levels were not proficient. 



1 Only for science and social studies. 

2 Only for reading/language arts and mathematics. 

S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Ohio 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 






✓ 


✓ 


State assessment staff 


✓ 


✓ 


✓ 


✓ 


State instruction and curriculum staff 








✓ 


Test vendor 


✓ 


✓ 


✓ 


✓ 


Outside experts 


✓ 


✓ 




✓ 


Special education teachers 






✓ 


✓ 


General education teachers 








✓ 


Content specialists 








✓ 


School psychologists/counselors 










School/district/state administrators 






✓ 




Parents 








✓ 



Validity 



Type of validity 


Types of evidence 




Scoring and reporting 
structures consistent with 
the subdomain 


Scoring and reporting 
documents 


✓ 


Relation to internal and 
external variables 


Correlational study 


✓ 


Purposes of the assessment 


Anecdotal or 
committee process 


✓ 


Intended and unintended 
consequences 


Not available for 
examination 


✓ 


Measurement of construct 
relevance 


Statistical analyses 




Grade-level equating 





Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

General link to state content standards 

Y Grade or grade span 

Y Expanded benchmarks 

Y Alternate indicators or tasks 



Reliability 



Type of reliability 


Types of evidence 


✓ 


Variability across groups 


NCLB group statistical 
analyses 


✓ 


Internal consistency of item 
responses 


Formal study or expert 
panel review 


✓ 


Interrater consistency in 
scoring 


Statistical analysis 
conducted as part of 
actual scoring 


✓ 


Standard errors of 
measurement 


Statistical analysis 


Other Types of Technical Quality 


Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Formal study 


✓ 


Ongoing/monitoring 


External alignment 
study 




Fairness 






Implementation processes 





S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Ohio 



Eligibility and Administration 



Eligibility 

Ohio provided the following guidelines to be used 
in determining when a child’s significant cognitive 
disability justified an alternate assessment based 
on alternate achievement standards. 

Had a severe cognitive disability 
Required modified instruction 

Required extensive support for skill 
generalization 

•A Required modified curriculum 
Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

•A Parent signature was required 

Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

•A Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

•A A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
•A Student work samples only 

Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 
•A As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

Teacher decided content to be assessed 
•A Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Ohio 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

Two scorers 

• Who scored? 

State or state-contracted scorer 

• How were scoring conflicts resolved? 

Y A third person adjudicated 

A third rater scored the assessment 

One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 

Y Ability to generalize across settings 

Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 

Y Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Y Work with nondisabled peers 

Appropriate human and technological 
supports 

None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 




Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 




✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
Y' Scores 

Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
■S Scores 

Y Standard/strand breakouts 
Indicator/benchmark breakouts 

Y Performance/achievement level descriptors 
Sample test items 



f \ 

State department of education 
website 

www. ode. state. oh. us 

V ) 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Ohio 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 




Percent 

□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Mathematics 




Percent 

□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts Mathematics 




Percent Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


1.78 


1.82 


6-8 


1.73 


1.74 


9-12 


1.44 


1.45 



1 Of all students who received valid scores on the specific types of accountability assessment. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 




Oklahoma 



Overview of the Oklahoma Alternate Assessment Program (OAAP) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Oklahoma Alternate 
Assessment Program (OAAP) were to 

Y Evaluate programs 

Y Guide classroom instruction 

Y Measure student progress/performance toward state standards 

Y Assess student access to content standards/general curriculum 

Y Assess individual students’ strengths/weaknesses 

Y Document academic achievement 

Y Measure student progress toward IEP goals 



Alternate Assessment 

Oklahoma Alternate Assessment Program (OAAP) 
Alternate Assessment Approaches 
Rating scale/checklist 

Y Portfolio/body of evidence 
Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 

Y Science 

Y Social studies 
Functional skills 

Grades Assessed 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 

Varied by teacher discretion 1 
Mathematics 

Varied by teacher discretion 2 

Who was involved in developing the alternate 
assessment? 

Assessment company 

Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 
Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Student’s classroom teacher 
School- or district-based educator 

Y State or state-contracted scorer 
Machine scored 
Paraprofessional 

1 The number of general content standards was 4. 

2 The number of general content standards was 5. 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Oklahoma 



Alternate Achievement Standards 

Alternate Achievement Descriptors 

Oklahoma developed 

•A The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

State did not report descriptors. 

Alternate Achievement Cut Scores 

Oklahoma developed 

Unique cut scores for each grade 
Unique cut scores for grade spans 
•A One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Oklahoma 
proficiency levels for 
alternate assessment 


Advanced 


Advanced 


Proficient 


Satisfactory 


Basic 


Limited Knowledge 




Unsatisfactory 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 
•A State instruction and curriculum staff 
Test vendor 
•A Outside experts 
•A Special education teachers 
•A General education teachers 
•A Content specialists 
•A School psychologists/counselors 
•A School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Reasoned Judgment standard-setting 
methodology. 



Students who performed at the Advanced and 
Satisfactory levels were proficient for purposes of 
the No Child Left Behind Act, whereas students 
who scored at the Limited Knowledge and 
Unsatisfactory levels were not proficient. 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Oklahoma 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 1 


Reliability 1 


Alignment 


Fairness 


State special education staff 






✓ 


✓ 


State assessment staff 






✓ 


✓ 


State instruction and curriculum staff 






✓ 


✓ 


Test vendor 










Outside experts 






✓ 


✓ 


Special education teachers 






✓ 


✓ 


General education teachers 






✓ 


✓ 


Content specialists 






✓ 


✓ 


School psychologists/counselors 








✓ 


School/district/state administrators 








✓ 


Parents 








✓ 



1 State reported “other” stakeholders. 



Validity Reliability 



Type of validity Types of evidence 

Scoring and reporting 
structures consistent with 
the subdomain 

Relation to internal and 
external variables 

Purposes of the assessment 

Intended and unintended 
consequences 

Measurement of construct 
relevance 

Grade-level equating 



Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 

Y Grade or grade span 
•Y Expanded benchmarks 

Alternate indicators or tasks 



Type of reliability Types of evidence 



Variability across groups 

Internal consistency of item 
responses 

■/ Interrater consistency in 
scoring 

Standard errors of 
measurement 


Training documents or 
anecdotal 


Other Types of Technical Quality 


Type of technical quality 


Types of evidence 


Alignment: 
■/ Initial 


Formal study 


•/ Ongoing/monitoring 


External alignment 


Fairness 

Implementation processes 


study 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Oklahoma 



Eligibility and Administration 



Eligibility 

Oklahoma provided the following guidelines to 
be used in determining when a child’s significant 
cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

Parent signature was required 

•A Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

•A Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

•A No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
•A Student work samples only 

Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 
•A As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

•A Teacher decided content to be assessed 
•A Teacher selected materials 

Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Oklahoma 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

Three or more scorers 

• Who scored? 

State or state-contracted scorer 

• How were scoring conflicts resolved? 

A third person adjudicated 
Y A third rater scored the assessment 

One person scored, or scores were 
combined 

Elements of student performance used in scoring 
Accuracy of student response 
Ability to generalize across settings 

Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 

Y Instruction in multiple settings 

Y Opportunities to plan, monitor, and evaluate 
work 

Y Work with nondisabled peers 

•Y Appropriate human and technological 
supports 

None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 




✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 

Y Scores 

Y Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 

Y Scores 

Standard/strand breakouts 
Indicator/benchmark breakouts 

Y Performance/achievement level descriptors 
Sample test items 



( \ 

State department of education 

website 

http ://sde . state . ok. u s/ 

V ) 



V Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Oklahoma 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated 

Reading/Language Arts 

Grades 




Percent 

□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 1 

□ Exempt/absent 



in state accountability assessments 

Mathematics 

Grades 




Percent 

□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 1 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 2 



Reading/Language Arts 




Percent 



Mathematics 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ Students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.90 


0.88 


6-8 


0.78 


0.78 


9-12 


0.65 


0.62 



M Missing. Information not provided by state. 

1 For this state this included an alternate assessment based on modified achievement standards. 

2 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 




Oregon 



Overview of the Oregon Assessment System: Extended Assessment (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Oregon Assessment 
System: Extended Assessment were to 

Evaluate programs 
Guide classroom instruction 

Measure student progress/performance toward state standards 

Y Assess student access to content standards/general curriculum 

Y Assess individual students’ strengths/weaknesses 

Y Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

Oregon Assessment System: Extended 
Assessment 

Alternate Assessment Approaches 

Rating scale/checklist 
Portfolio/body of evidence 

Y Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 

Y Science 
Social studies 

Y Functional skills 
Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

Y More than 1 month to 2 months 
More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
Varied by grade level 
Mathematics 

5 of 5 standards 

Who was involved in developing the alternate 
assessment? 

Assessment company 

Y Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 
Parents 
Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

Y A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Student’s classroom teacher 
School- or district-based educator 

Y State or state-contracted scorer 
Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Oregon 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 
•A State instruction and curriculum staff 
Test vendor 
•A Outside experts 
•A Special education teachers 
•A General education teachers 
■A Content specialists 

School psychologists/counselors 
School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Bookmark or Item Mapping standard- 
setting methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Oregon 
proficiency levels for 
alternate assessment 


Advanced 


Exceeds Standard 


Proficient 


Meets Standard 


Basic 


Does Not Yet Meet Standard 



Alternate Achievement Descriptors 

Oregon developed 

The same descriptors that applied to all 
grades tested 

•A The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

A example of the Meets Standard descriptor in 
reading/language arts is as follows: 

• Students’ scores at this level indicate a 
frequent understanding of reduced depth, 
breadth, and complexity of items and 
relatively consistent academic performance. 

Alternate Achievement Cut Scores 

Oregon developed 

Unique cut scores for each grade 
•A Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Exceeds Standard 
and Meets Standard levels were proficient for 
purposes of the No Child Left Behind Act, whereas 
students who scored at the Does Not Yet Meet 
Standard level were not proficient. 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Oregon 



Technical Quality 



Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


✓ 






✓ 


State assessment staff 


✓ 




✓ 


✓ 


State instruction and curriculum staff 


✓ 




✓ 


✓ 


Test vendor 




✓ 




✓ 


Outside experts 


✓ 




✓ 




Special education teachers 


✓ 




✓ 




General education teachers 


✓ 




✓ 




Content specialists 


✓ 




✓ 




School psychologists/counselors 










School/district/state administrators 










Parents 


✓ 









Validity 



Reliability 



Type of validity 



Types of evidence Type of reliability 



Types of evidence 



•/ Scoring and reporting 
structures consistent with 
the subdomain 

Relation to internal and 
external variables 

Purposes of the assessment 

✓ Intended and unintended 
consequences 



Measurement of construct 
relevance 

Grade-level equating 



Scoring and reporting 
documents 



Public reports, post 
hoc data collection/ 
analysis 



Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 

Y Grade or grade span 

Y Expanded benchmarks 

Y Alternate indicators or tasks 



Variability across groups 

Internal consistency of item 
responses 

Interrater consistency in 
scoring 

Standard errors of 
measurement 



Other Types of Technical Quality 


Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Anecdotal or 
committee process 


✓ 


Ongoing/monitoring 


Internal and external 
alignment studies 


✓ 


Fairness 


Regularly scheduled 
bias review by experts, 
statistical analyses 


✓ 


Implementation processes 


Training, 
administration manual/ 
guide, monitoring, post 
hoc data collection/ 
analysis 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Oregon 



Eligibility and Administration 



Eligibility 

Oregon provided the following guidelines to be 
used in determining when a child’s significant 
cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

•A Parent signature was required 

•A Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

•A All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

•A No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

•A A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

•A State required standardized tasks/test items/ 
rating scales 

State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

Combination of work samples and other 
evidence 

•A No student work samples 

Assessment was administered 

As part of day-to-day student instruction 
Separately from student’s daily work 

•A Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

Teacher decided content to be assessed 
Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Oregon 



Scoring and Reporting 
Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

One scorer 

• Who scored? 

State or state-contracted scorer 

• How were scoring conflicts resolved? 

A third person adjudicated 
A third rater scored the assessment 

Y One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 
Ability to generalize across settings 
Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

Y None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 


✓ 


✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
Y' Scores 

Y Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
Y' Scores 

Standard/strand breakouts 
Indicator/benchmark breakouts 
Performance/achievement level descriptors 
Sample test items 



f \ 

State department of education 

website 

www.ode.state.or.us/ 

V ) 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Oregon 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 




Percent 

□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Mathematics 




Percent 

□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts 




Percent 



Mathematics 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


3.17 


2.14 


6-8 


1.67 


1.28 


9-12 


1.27 


1.05 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 



Pennsylvania 



Overview of the Pennsylvania Alternate System of Assessment (PASA) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purpose of the 2006-07 Pennsylvania Alternate 
System of Assessment (PASA) was to 

Evaluate programs 
Guide classroom instruction 

•A Measure student progress/performance toward state standards 
Assess student access to content standards/general curriculum 
Assess individual students’ strengths/weaknesses 
Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

Pennsylvania Alternate System of Assessment 
(PASA) 

Alternate Assessment Approaches 

Rating scale/checklist 
Portfolio/body of evidence 

Y Performance task/events 

Y Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 
Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

Y More than 1 month to 2 months 
More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
1 of 8 standards 
Mathematics 

7 of 11 standards 

Who was involved in developing the alternate 
assessment? 

Assessment company 

Y Research company/university/independent 
researcher 

Technical assistance provider 

State personnel 

Parents 

Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

Y A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Student’s classroom teacher 
School- or district-based educator 

Y State or state-contracted scorer 
Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Pennsylvania 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
State assessment staff 
State instruction and curriculum staff 
•A Test vendor 
•A Outside experts 
•A Special education teachers 
General education teachers 
•A Content specialists 

School psychologists/counselors 
•A School/district/state administrators 
Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Modified Angoff standard-setting 
methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Pennsylvania 
proficiency levels for 
alternate assessment 


Advanced 


Advanced 


Proficient 


Proficient 


Basic 


Novice 




Emerging 



Students who performed at the Advanced and 
Proficient levels were proficient for purposes of the 
No Child Left Behind Act, whereas students who 
scored at the Novice and Emerging levels were not 
proficient. 



Alternate Achievement Descriptors 

Pennsylvania developed 

The same descriptors that applied to all 
grades tested 

•A The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Proficient descriptor (Level A) in 

reading/language arts (grade 5-6) is as follows: 

• The student is able to make appropriate 
discriminations such as matching, selecting 
objects, identifying categories of objects, 
recognizing and demonstrating the function 
of objects, as well as respond to ‘who’ and 
‘what’ listening comprehension questions 
by selecting, independently or with some 
support that maintains the cognitive 
complexity of the skills. Correct answers 
and choices consist of objects that the 
student is likely to have direct experience 
with in his/her environment. Often, two 
of the three objects included among the 
choices are similar to one another in terms 
of appearance and function, making the 
discrimination more challenging. 

Pennsylvania developed 

Unique cut scores for each grade 

•A Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Pennsylvania 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders Validity Reliability Alignment Fairness 

State special education staff 

State assessment staff 

State instruction and curriculum staff 



Test vendor 


✓ 


✓ 


✓ 


✓ 


Outside experts 


✓ 


✓ 


✓ 


✓ 


Special education teachers 


✓ 


✓ 


✓ 


✓ 


General education teachers 


✓ 


✓ 






Content specialists 


✓ 


✓ 


✓ 


✓ 


School psychologists/counselors 










School/district/state administrators 


✓ 


✓ 


✓ 


✓ 


Parents 


✓ 









Validity Reliability 



Type of validity Types of evidence 

Scoring and reporting 
structures consistent with 
the subdomain 

Relation to internal and 
external variables 

■/ Purposes of the assessment Anecdotal or 

committee process 

■/ Intended and unintended Survey 

consequences 

■/ Measurement of construct Anecdotal or 

relevance committee process 

■/ Grade-level equating Formal study or expert 

panel review 



Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

•A General link to state content standards 
A Grade or grade span 
A Expanded benchmarks 
A Alternate indicators or tasks 



Type of reliability Types of evidence 





Variability across groups 




✓ 


Internal consistency of item 
responses 


Formal study or expert 
panel review 


✓ 


Interrater consistency in 
scoring 


Statistical analysis 
conducted as part of 
training and actual 
scoring 




Standard errors of 
measurement 




Other Types of Technical Quality 


Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Formal study 


✓ 


Ongoing/monitoring 


External alignment 
study 


✓ 


Fairness 


Regularly scheduled 
bias review by experts 


✓ 


Implementation processes 


Training, 
administration manual/ 
guide 



V Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Pennsylvania 



Eligibility and Administration 



Eligibility 

Pennsylvania provided the following guidelines 
to be used in determining when a child’s 
significant cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. 

Assessment Content Selection 

State determined 

•A All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

•A An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

•A A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

•A State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

Combination of work samples and other 
evidence 

•A No student work samples 

Assessment was administered 

As part of day-to-day student instruction 
•A Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

•A Teacher decided content to be assessed 
Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Pennsylvania 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

Two scorers 

• Who scored? 

State or state-contracted scorer 

• How were scoring conflicts resolved? 

Y A third person adjudicated 

A third rater scored the assessment 

One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 
Ability to generalize across settings 

Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Y Appropriate human and technological 
supports 

None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 




Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 







Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

State’s achievement standards 
Y' Scores 
Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 
Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
■S Scores 

Standard/strand breakouts 
•A Indicator/benchmark breakouts 
A Performance/achievement level descriptors 
Sample test items 



f \ 

State department of education 

website 

www.pasaassessment.org 

V ) 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Pennsylvania 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 




Percent 



Mathematics 




Percent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts 




Percent 



Grades 



9-12 



Mathematics 

165 




T 1 1 1 1 

0 20 40 60 80 100 
Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.91 


1.02 


6-8 


0.93 


1.02 


9-12 


0.82 


0.88 



1 Of all students who received valid scores on the specific types of accountability assessment. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 




Rhode Island 



Overview of the Rhode Island Alternate Assessment (RIAA) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Rhode Island Alternate 
Assessment (RIAA) were to 

Y Evaluate programs 

Y Guide classroom instruction 

Y Measure student progress/performance toward state standards 

Y Assess student access to content standards/general curriculum 
Assess individual students’ strengths/weaknesses 

Y Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

Rhode Island Alternate Assessment (RIAA) 
Alternate Assessment Approaches 
Rating scale/checklist 

Y Portfolio/body of evidence 
Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 
Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
2 of 8 standards 
Mathematics 

2 of 4 standards 

Who was involved in developing the alternate 
assessment? 

Y Assessment company 

Y Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 

Y Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Student’s classroom teacher 
School- or district-based educator 

Y State or state-contracted scorer 
Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Rhode Island 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 
•A State instruction and curriculum staff 
•A Test vendor 
Outside experts 
•A Special education teachers 
•A General education teachers 
•A Content specialists 

School psychologists/counselors 
•A School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Performance Profile Method and 
the Reasoned Judgment standard-setting 
methodologies. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Rhode Island 
proficiency levels for 
alternate assessment 


Advanced 


Proficient with Distinction 


Proficient 


Proficient 


Basic 


Partially Proficient 



Alternate Achievement Descriptors 

Rhode Island developed 

The same descriptors that applied to all 
grades tested 

•A The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Proficient descriptor in reading/ 
language arts (grades 3-8) is as follows: 

• Students performing at this level submitted 
datafolios that demonstrate: 

- consistent connections to the grade level 
content strands through participation 

in instructional activities throughout the 
year that are consistently aligned with 
the Word Identification and Vocabulary 
and Initial Understanding, Analysis and 
Interpretation of Literacy or Information 
Text AAGSEs. 

- participation in distinct standards based 
instructional activities that demonstrate 
consistent application of the AAGSEs 
across most entries within the context of 
the Structured Performance Tasks. 

- consistent progress during the year. 

- adequate level of accuracy in 
instructional activities and/or 

- adequate level of independence 
completing instructional activities. 



Students who performed at the Proficient with 

Distinction and Proficient levels were proficient for Alternate Achievement Cut Scores 
purposes of the No Child Left Behind Act, whereas 

students who scored at the Partially Proficient level Rhode Island developed 

were not proficient. Unique cut scores for each grade 

•A Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Rhode Island 



Technical Quality 



Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


✓ 


✓ 




✓ 


State assessment staff 


✓ 


✓ 




✓ 


State instruction and curriculum staff 


✓ 








Test vendor 


✓ 


✓ 




✓ 


Outside experts 


✓ 


✓ 


✓ 




Special education teachers 


✓ 


✓ 


✓ 


✓ 


General education teachers 


✓ 


✓ 


✓ 


✓ 


Content specialists 


✓ 




✓ 




School psychologists/counselors 










School/district/state administrators 


✓ 




✓ 


✓ 


Parents 


✓ 






✓ 



Validity 



Type of validity Types of evidence 



Scoring and reporting 
structures consistent with 
the subdomain 

Relation to internal and 
external variables 




Purposes of the assessment 


✓ Intended and unintended 


Survey, public 


consequences 


reports, post hoc data 
collection/analysis 


✓ Measurement of construct 


Statistical analyses, 


relevance 

Grade-level equating 


construct analyses 



Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 

Y Grade or grade span 

Y Expanded benchmarks 

Y Alternate indicators or tasks 



Reliability 



Type of reliability 


Types of evidence 


Variability across groups 

Internal consistency of item 
responses 




■/ Interrater consistency in 


Statistical analysis 


scoring 

Standard errors of 
measurement 


conducted as part of 
actual scoring 


Other Types of Technical Quality 


Type of technical quality 


Types of evidence 



Alignment: 



✓ 


Initial 


Formal study 


✓ 


Ongoing/monitoring 


Other alignment 






studies 


✓ 


Fairness 


Regularly scheduled 






bias review by experts 


✓ 


Implementation processes 


Training, 



administration manual/ 
guide, monitoring, post 
hoc data collection/ 



analysis 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Rhode Island 



Eligibility and Administration 



Eligibility 

Rhode Island provided the following guidelines 
to be used in determining when a child’s 
significant cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

•A Parent signature was required 

•A Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

•A Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
•A Student work samples only 

Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 
•A As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

•A Teacher decided content to be assessed 
•A Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Rhode Island 



Scoring and Reporting 
Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

Two scorers 

• Who scored? 

State or state-contracted scorer 

• How were scoring conflicts resolved? 

A third person adjudicated 
Y A third rater scored the assessment 

One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 

•Y Ability to generalize across settings 
•Y Amount of independence 

Y Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

Y None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 




✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 

Y Scores 
Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 

Y Scores 

Y Standard/strand breakouts 
Indicator/benchmark breakouts 
Performance/achievement level descriptors 
Sample test items 



f \ 

State department of education 

website 

www.ride.ri.gov/assessment 

V ) 



■/ Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Rhode Island 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Mathematics 



i^raaes 



Grades 

3 2 


3 2 


1 94 n 3-5 


1 94 1 


3 3 


3 3 


1 94 n 6-8 


i 94 n 






1 92 41 9-12 


h 92 HI 


1 1 1 1 1 


1 1 1 1 1 



0 20 40 60 80 100 0 20 40 60 80 100 



Percent 



Percent 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts 




Percent 



Mathematics 




0 20 40 60 80 100 



Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.35 


0.35 


6-8 


0.35 


0.35 


9-12 


0.20 


0.17 



M Missing. Information not provided by state. 

1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 



South Carolina 



Overview of the South Carolina Alternate Assessment (SC-Alt) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 South Carolina Alternate 
Assessment (SC-Alt) were to 

Evaluate programs 

Y Guide classroom instruction 

Y Measure student progress/performance toward state standards 
Assess student access to content standards/general curriculum 
Assess individual students’ strengths/weaknesses 
Document academic achievement 

Measure student progress toward IEP goals 



Alternate Assessment 

South Carolina Alternate Assessment (SC-Alt) 
Alternate Assessment Approaches 
Rating scale/checklist 
Portfolio/body of evidence 

Y Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 

Y Science 

Y Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

Y More than 1 month to 2 months 
More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
3 of 4 standards 
Mathematics 

5 of 5 standards 

Who was involved in developing the alternate 
assessment? 

Assessment company 

Y Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 
Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Y Student’s classroom teacher 
School- or district-based educator 
State or state-contracted scorer 

Y Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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South Carolina 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 
•A State instruction and curriculum staff 
•A Test vendor 
Outside experts 
•A Special education teachers 
•A General education teachers 
•A Content specialists 

School psychologists/counselors 
•A School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Item-Descriptor Matching standard- 
setting methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding South Carolina 
proficiency levels for 
alternate assessment 


Advanced 


Level 4 


Proficient 


Level 3 


Basic 


Level 2 




Level 1 



Students who performed at Level 4 and Level 3 
were proficient for purposes of the No Child Left 
Behind Act, whereas students who scored at 
Level 2 and Level 1 were not proficient. 



Alternate Achievement Descriptors 

South Carolina developed 

The same descriptors that applied to all 
grades tested 

•A The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Level 3 descriptor in reading/ 
language arts (grades 3-5) is as follows: 

• Students performing at Level 3 

demonstrate increasing academic skills and 
competencies. 

They should be able to: 

- identify story elements in text (e.g., 
characters, settings, events, cause and 
effect, and problem solution); 

- read words and simple sentences; 

- generate an idea and use words, 
pictures, or oral language to write; 

- follow one-step oral or signed directions; 

- communicate agreement or 
disagreement appropriately. 

Alternate Achievement Cut Scores 

South Carolina developed 

Unique cut scores for each grade 
•A Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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South Carolina 



Technical Quality 



Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 








✓ 


State assessment staff 


✓ 


✓ 




✓ 


State instruction and curriculum staff 








✓ 


Test vendor 


✓ 


✓ 


✓ 


✓ 


Outside experts 






✓ 




Special education teachers 






✓ 


✓ 


General education teachers 






✓ 


✓ 


Content specialists 






✓ 


✓ 


School psychologists/counselors 










School/district/state administrators 






✓ 


✓ 


Parents 








✓ 



Validity 



Type of validity 


Types of evidence 


✓ 


Scoring and reporting 
structures consistent with 
the subdomain 


Scoring and reporting 
documents 


✓ 


Relation to internal and 
external variables 


Item-to-item analysis 


✓ 


Purposes of the assessment 


Alignment study, 
construct validity 
analysis 




Intended and unintended 
consequences 




✓ 


Measurement of construct 
relevance 


Statistical and 
construct analyses 




Grade-level equating 





Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 
•Y Grade or grade span 

Y Expanded benchmarks 
Alternate indicators or tasks 



Reliability 



Type of reliability 


Types of evidence 


✓ 


Variability across groups 


NCLB group statistical 
analyses, review of 
disability group results 


✓ 


Internal consistency of item 
responses 


Formal study or expert 
panel review 


✓ 


Interrater consistency in 
scoring 


Statistical analysis 
conducted as part of 
actual scoring 


✓ 


Standard errors of 
measurement 


Statistical analysis 



Other Types of Technical Quality 



Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Anecdotal or 
committee process 


✓ 


Ongoing/monitoring 


External alignment 
study 


✓ 


Fairness 


Regularly scheduled 
bias review by experts, 
statistical analyses 


✓ 


Implementation processes 


Training, 
administration manual/ 
guide 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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South Carolina 



Eligibility and Administration 



Eligibility 

South Carolina provided the following guidelines 
to be used in determining when a child’s 
significant cognitive disability justified an alternate 
assessment based on alternate achievement 
standards 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

•A Parent signature was required 

•A Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

•A All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

•A An observer/monitor who was present 

A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

A State required standardized tasks/test items/ 
rating scales 

State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

Combination of work samples and other 
evidence 

A No student work samples 

Assessment was administered 

As part of day-to-day student instruction 
A Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

Teacher decided content to be assessed 
Teacher selected materials 

A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

A Teacher interpreted/recorded student 
responses or scores 



A Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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South Carolina 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

One scorer 

• Who scored? 

Student’s classroom teacher, machine 
scored 

• How were scoring conflicts resolved? 

A third person adjudicated 
A third rater scored the assessment 

Y One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 
Ability to generalize across settings 
Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

Y None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 


✓ 


✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
Y' Scores 

Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
■S Scores 

Standard/strand breakouts 
Indicator/benchmark breakouts 

Y Performance/achievement level descriptors 
Sample test items 



f N 

State department of education 
website 

www.ed.sc.gov/ 

V ) 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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South Carolina 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 




Percent 

□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Mathematics 




Percent 

□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts 




Percent 



Mathematics 




n 



0 20 40 60 80 100 



Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ Students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.61 


0.53 


6-8 


0.41 


0.37 


9-12 


0.47 


0.43 



M Missing. Information not provided by state. 

1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, EDFacfs - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 




South Dakota 



Overview of the Dakota State Test of Educational Progress-Alternate (DSTEP-A) 
( 2006 - 07 ) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purpose of the 2006-07 Dakota State Test of 
Educational Progress-Alternate (DSTEP-A) was to 

Evaluate programs 
Guide classroom instruction 

Y Measure student progress/performance toward state standards 
Assess student access to content standards/general curriculum 
Assess individual students’ strengths/weaknesses 
Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

Dakota State Test of Educational Progress- 
Alternate (DSTEP-A) 

Alternate Assessment Approaches 

Y Rating scale/checklist 

Y Portfolio/body of evidence 
Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 

Y Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

Y More than 1 month to 2 months 
More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 

4 of 4 standards 
Mathematics 

5 of 5 standards 

Who was involved in developing the alternate 
assessment? 

Y Assessment company 

Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 
Parents 
Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

Y A certified educator who was not the student’s 
teacher 

Y Paraprofessional 

Who scored the alternate assessment? 

Student’s classroom teacher 

Y School- or district-based educator 

Y State or state-contracted scorer 
Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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South Dakota 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 
•A State instruction and curriculum staff 
Test vendor 
•A Outside experts 
•A Special education teachers 
•A General education teachers 
■A Content specialists 

School psychologists/counselors 
School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Modified Angoff standard-setting 
methodology. 



Alternate Achievement Descriptors 

South Dakota developed 

The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

•A Descriptors that were unique for each grade 
tested 

An example of the Applying descriptor in reading/ 
language arts is as follows: 

• Use context to understand the meaning of 
words; identify main idea and supporting 
details (direct meaning); use the text 
structures to create meaning; describe 
similes and personification as types of 
figurative language; describe various cultures 
and time periods in text; retrieve information 
from a variety of references; compare 
information on one topic contained in several 
sources. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding South Dakota 
proficiency levels for 
alternate assessment 


Advanced 


Advancing 


Proficient 


Applying 


Basic 


Developing 




Introducing 



Alternate Achievement Cut Scores 

South Dakota developed 
•A Unique cut scores for each grade 
Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Advancing and 
Applying levels were proficient for purposes of the 
No Child Left Behind Act, whereas students who 
scored at the Developing and Introducing levels 
were not proficient. 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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South Dakota 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


✓ 


✓ 


✓ 


✓ 


State assessment staff 


✓ 


✓ 


✓ 


✓ 


State instruction and curriculum staff 




✓ 


✓ 


✓ 


Test vendor 


✓ 


✓ 


✓ 


✓ 


Outside experts 


✓ 


✓ 


✓ 


✓ 


Special education teachers 






✓ 


✓ 


General education teachers 






✓ 


✓ 


Content specialists 








✓ 


School psychologists/counselors 










School/district/state administrators 










Parents 








✓ 



Validity 



Type of validity 


Types of evidence 




Scoring and reporting 
structures consistent with 
the subdomain 




✓ 


Relation to internal and 
external variables 


Not available for 
examination 


✓ 


Purposes of the assessment 


Alignment study 


✓ 


Intended and unintended 
consequences 


Post hoc data 
collection/analysis 


✓ 


Measurement of construct 
relevance 


Not available for 
examination 


✓ 


Grade-level equating 


Formal study or expert 
panel review 



Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 

Y Grade or grade span 
•Y Expanded benchmarks 

Y Alternate indicators or tasks 



Reliability 



Type of reliability 


Types of evidence 




Variability across groups 




✓ 


Internal consistency of item 
responses 


Not available for 
examination 


✓ 


Interrater consistency in 
scoring 


Training documents or 
anecdotal 


✓ 


Standard errors of 
measurement 


Statistical analysis 


Other Types of Technical Quality 


Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Formal study 


✓ 


Ongoing/monitoring 


External alignment 
study 


✓ 


Fairness 


Regularly scheduled 
bias review by experts, 
statistical analyses 


✓ 


Implementation processes 


Training, 
administration manual/ 
guide, monitoring 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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South Dakota 



Eligibility and Administration 



Eligibility 

South Dakota provided the following guidelines 
to be used in determining when a child’s 
significant cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. 

Assessment Content Selection 

State determined 

•A All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

•A A certified educator who was not the 
student’s teacher 

•A Paraprofessional 

Procedures followed in gathering performance 
evidence included 

•A State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

•A Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 
•A As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

Teacher decided content to be assessed 
•A Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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South Dakota 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

Three or more scorers 

• Who scored? 

School- or district-based educator, state or 
state-contracted scorer 

• How were scoring conflicts resolved? 

A third person adjudicated 
Y A third rater scored the assessment 

One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 

Y Ability to generalize across settings 

Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 

Y Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Y Work with nondisabled peers 

Appropriate human and technological 
supports 

None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 


✓ 


✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 

Y Scores 
Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 

Y Scores 

Standard/strand breakouts 
Indicator/benchmark breakouts 

Y Performance/achievement level descriptors 
Sample test items 



( ^ 

State department of education 

website 

http://doe.sd.gov/ 

V ) 



■/ Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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South Dakota 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Mathematics 



i^raaes Grades 





Percent 



Percent 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts 




Percent 



Mathematics 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.62 


0.49 


6-8 


0.57 


0.46 


9-12 


0.44 


0.40 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 



264 



National Study on Alternate Assessments: State Profiles 



Tennessee 



Overview of the Tennessee Comprehensive Assessment Program-Alternate 
Portfolio (TCAP-Alt Portfolio) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Tennessee 
Comprehensive Assessment Program-Alternate Portfolio (TCAP-Alt Portfolio) were to 

Evaluate programs 

Y Guide classroom instruction 

Y Measure student progress/performance toward state standards 

Y Assess student access to content standards/general curriculum 

Y Assess individual students’ strengths/weaknesses 

Y Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

Tennessee Comprehensive Assessment Program- 
Alternate Portfolio (TCAP-Alt Portfolio) 

Alternate Assessment Approaches 

Rating scale/checklist 

Y Portfolio/body of evidence 
Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 

Y Science 

Y Social studies 
Functional skills 

Grades Assessed 

yyyyyyyyy 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
3 of 3 standards 
Mathematics 

3 of 5 standards 

Who was involved in developing the alternate 
assessment? 

Assessment company 

Research company/university/independent 
researcher 

Y Technical assistance provider 

Y State personnel 

Y Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Student’s classroom teacher 
School- or district-based educator 

Y State or state-contracted scorer 
Machine scored 
Paraprofessional 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Tennessee 



Alternate Achievement Standards 

Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 
•A State instruction and curriculum staff 
Test vendor 
•A Outside experts 
•A Special education teachers 
•A General education teachers 
•A Content specialists 
•A School psychologists/counselors 
•A School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

The state did not report a standard-setting 
methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Tennessee 
proficiency levels for 
alternate assessment 


Advanced 


Advanced 


Proficient 


Proficient 


Basic 


Below Proficient 



Students who performed at the Advanced and 
Proficient levels were proficient for purposes of 
the No Child Left Behind Act, whereas students 
who scored at the Below Proficient level were not 
proficient. 



Alternate Achievement Descriptors 

Tennessee developed 

•A The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Proficient descriptor in reading/ 
language arts is as follows: 

• Student’s program offers opportunities for 
standards-based activities that are age 
appropriate and functional, as well as 
some opportunities to make choices within 
activities, receive support from special 
education personnel and others, receive 
instruction in multiple settings (some of which 
are inclusive), and interact with non-disabled 
peers in some situations. 

Alternate Achievement Cut Scores 

Tennessee developed 

Unique cut scores for each grade 
Unique cut scores for grade spans 
•A One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Tennessee 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


✓ 


✓ 


✓ 


✓ 


State assessment staff 


✓ 


✓ 


✓ 


✓ 


State instruction and curriculum staff 


✓ 




✓ 


✓ 


Test vendor 




✓ 






Outside experts 


✓ 




✓ 


✓ 


Special education teachers 


✓ 


✓ 


✓ 


✓ 


General education teachers 


✓ 






✓ 


Content specialists 


✓ 


✓ 


✓ 


✓ 


School psychologists/counselors 




✓ 






School/district/state administrators 




✓ 






Parents 


✓ 


✓ 


✓ 


✓ 



Validity 



Type of validity Types of evidence 

■/ Scoring and reporting Scoring and reporting 

structures consistent with documents 

the subdomain 
Relation to internal and 
external variables 

Purposes of the assessment 

Intended and unintended 
consequences 

✓ Measurement of construct Not available for 

relevance examination 

■/ Grade-level equating Not available for 

examination 



Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 

Y Grade or grade span 

Y Expanded benchmarks 

Y Alternate indicators or tasks 



Reliability 



Type of reliability Types of evidence 

Variability across groups 

Internal consistency of item 
responses 

■/ Interrater consistency in Statistical analysis 

scoring conducted as part of 

actual scoring 

Standard errors of 
measurement 



Other Types of Technical Quality 



Type of technical quality Types of evidence 





Alignment: 




✓ 


Initial 

Ongoing/monitoring 


Anecdotal or 
committee process 


✓ 


Fairness 


Bias review, not 
conducted regularly 


✓ 


Implementation processes 


Post hoc data 
collection/analysis 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Tennessee 



Eligibility and Administration 



Eligibility 

Tennessee provided the following guidelines 
to be used in determining when a child’s 
significant cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

Parent signature was required 

•A Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

•A Most component - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
•A Student work samples only 

Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 
•A As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

•A Teacher decided content to be assessed 
•A Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Tennessee 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

Two scorers 

• Who scored? 

State or state-contracted scorer 

• How were scoring conflicts resolved? 

A third person adjudicated 
Y A third rater scored the assessment 

One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 

Y Ability to generalize across settings 
Amount of independence 

Y Amount of progress 

Environmental elements used in scoring 

Y Instruction in multiple settings 

Y Opportunities to plan, monitor, and evaluate 
work 

Y Work with nondisabled peers 

Y Appropriate human and technological 
supports 

None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 




✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
Y' Scores 

Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
Y' Scores 

Standard/strand breakouts 
Indicator/benchmark breakouts 
y' Performance/achievement level descriptors 
Sample test items 



( N 

State department of education 
website 

www.state.tn . us/ed ucation/ 

V J 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Tennessee 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments: 



Reading/Language Arts 



Mathematics 



Grades 



0 



n 


92 


1 






1 


i 


91 


II 


2 




2 


1 


96 


II 









0 20 40 60 80 100 



Percent 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 




Percent 

□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments : 1 



Reading/Language Arts 




Percent 



Mathematics 




0 20 40 60 80 100 



Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ Students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.86 


0.85 


6-8 


1.00 


0.99 


9-12 


0.15 


0.54 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 



Texas 



Overview of the Texas Assessment of Knowledge and Skills-Alternate (TAKS-Alt) 
( 2006 - 07 ) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Texas Assessment of 
Knowledge and Skills-Alternate (TAKS-Alt) were to 

Y Evaluate programs 

•Y Guide classroom instruction 

•Y Measure student progress/performance toward state standards 

Y Assess student access to content standards/general curriculum 

Y Assess individual students’ strengths/weaknesses 

Y Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

Texas Assessment of Knowledge and Skills- 
Alternate (TAKS-Alt) 

Alternate Assessment Approaches 

Rating scale/checklist 

Y Portfolio/body of evidence 

Y Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 

Y Science 

Y Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓✓✓✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
Varied by grade level 
Mathematics 

Varied by grade level 

Who was involved in developing the alternate 
assessment? 

Y Assessment company 

Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 

Y Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Y Student’s classroom teacher 
School- or district-based educator 
State or state-contracted scorer 
Machine scored 
Paraprofessional 



V Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

Texas Alternate Assessment for the 2006-07 School Year 
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Texas 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 

•A State assessment staff 

•A State instruction and curriculum staff 

•A Test vendor 

•A Outside experts 

•A Special education teachers 

•A General education teachers 

•A Content specialists 

•A School psychologists/counselors 

•A School/district/state administrators 

•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Body of Work standard-setting 
methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Texas 
proficiency levels for 
alternate assessment 


Advanced 


Commended 


Proficient 


Met Standard 


Basic 


Did Not Meet Standard 



Students who performed at the Commended and 
Met Standard levels were proficient for purposes 
of the No Child Left Behind Act, whereas students 
who scored at the Did Not Meet Standard level 
were not proficient. 



Alternate Achievement Descriptors 

Texas developed 

The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

•A Descriptors that were unique for each grade 
tested 

An example of the Met Standard descriptor in 
reading/language arts (grade 3) is as follows: 

• Satisfactory academic achievement; 
moderate success at a level that was at 
or somewhat above the state passing 
standard and may have required some 
additional cueing and prompting; may have 
generalized skills in a different context; 
sufficient understanding of the knowledge 
and skills that are linked to reading content 
measured at this grade. Students who met 
the standard: 

1. Begin to recognize words/symbols 

2. Self-select books 

3. Begin to use or understand some new 
vocabulary 

4. Identify one important idea in a text 

5. Identify one story element, such as 
characters, setting, or problem 

6. Begin to identify the purpose of a text 

7. Anticipate what comes next in a text 

8. Identify a cause or an effect when 
provided one part of the cause/effect 
relationship 

9. Begin to locate information in the text 
to answer questions 

10. Begin to use reading strategies in 
other contexts 



Alternate Achievement Cut Scores 

Texas developed 

Unique cut scores for each grade 
Unique cut scores for grade spans 
•A One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 

v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Texas 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


✓ 


✓ 


✓ 


✓ 


State assessment staff 


✓ 


✓ 


✓ 


✓ 


State instruction and curriculum staff 


✓ 


✓ 


✓ 


✓ 


Test vendor 


✓ 


✓ 


✓ 


✓ 


Outside experts 


✓ 


✓ 


✓ 


✓ 


Special education teachers 


✓ 


✓ 


✓ 


✓ 


General education teachers 


✓ 


✓ 


✓ 


✓ 


Content specialists 


✓ 


✓ 


✓ 


✓ 


School psychologists/counselors 


✓ 


✓ 


✓ 


✓ 


School/district/state administrators 


✓ 


✓ 


✓ 


✓ 


Parents 


✓ 


✓ 


✓ 


✓ 



Validity Reliability 



Type of validity Types of evidence 



■/ Scoring and reporting 


Scoring and reporting 


structures consistent with 


documents 


the subdomain 




Relation to internal and 




external variables 




Purposes of the assessment 




■/ Intended and unintended 


Not available for 


consequences 


examination 


Measurement of construct 




relevance 




Grade-level equating 





Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 

Y Grade or grade span 

Y Expanded benchmarks 
Alternate indicators or tasks 



Type of reliability 


Types of evidence 


Variability across groups 




Internal consistency of item 




responses 




■/ Interrater consistency in 


Not available for 


scoring 


examination 


Standard errors of 




measurement 





Other Types of Technical Quality 



Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Formal study 


✓ 


Ongoing/monitoring 


External alignment 
study 


✓ 


Fairness 


Bias review, not 
conducted regularly 



Implementation processes 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Texas 



Eligibility and Administration 



Eligibility 

Texas provided the following guidelines to be used 
in determining when a child’s significant cognitive 
disability justified an alternate assessment based 
on alternate achievement standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. 

Assessment Content Selection 

State determined 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

•A Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

•A No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

•A Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

•A Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 

As part of day-to-day student instruction 
Separately from student’s daily work 

•A Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

•A Teacher decided content to be assessed 
•A Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Texas 



Scoring and Reporting 
Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 1 
One scorer 

• Who scored? 

Student’s classroom teacher 

• How were scoring conflicts resolved? 

A third person adjudicated 
A third rater scored the assessment 

Y One person scored, or scores were 
combined 

Elements of student performance used in scoring 
Accuracy of student response 

Y Ability to generalize across settings 

Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Y Appropriate human and technological 
supports 

None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 







Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
Y' Scores 

Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
Y' Scores 

Standard/strand breakouts 
Indicator/benchmark breakouts 
Performance/achievement level descriptors 
Sample test items 



r \ 

State department of education 
website 

www.tea.state.tx.us/student.assessment 

V ) 



1 Portion of sample double scored. 

Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Texas 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Mathematics 





Percent 



Percent 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 1 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 1 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 2 



Reading/Language Arts 




Percent 



Mathematics 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


3.11 


2.64 


6-8 


3.28 


2.97 


9-12 


2.67 


2.91 



M Missing. Information not provided by state. 

1 For this state this included an alternate assessment based on grade-level or modified achievement standards. 

2 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, EDFacfs - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 



Utah 



Overview of Utah’s Alternate Assessment (UAA) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Utah’s Alternate 
Assessment (UAA) were to 

Y Evaluate programs 

Guide classroom instruction 

Y Measure student progress/performance toward state standards 
Assess student access to content standards/general curriculum 
Assess individual students’ strengths/weaknesses 
Document academic achievement 

Measure student progress toward IEP goals 



Alternate Assessment 

Utah’s Alternate Assessment (UAA) 
Alternate Assessment Approaches 
Rating scale/checklist 
Portfolio/body of evidence 

Y Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 

Y Science 
Social studies 
Functional skills 

Grades Assessed 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

Y More than 1 month to 2 months 
More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
1 standard 1 
Mathematics 
1 standard 1 

Who was involved in developing the alternate 
assessment? 

Assessment company 

Y Research company/university/independent 
researcher 

Y Technical assistance provider 

Y State personnel 

Y Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Y Student’s classroom teacher 
School- or district-based educator 
State or state-contracted scorer 
Machine scored 
Paraprofessional 

1 The number of general content standards varied by grade 
level. 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Utah 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 

State instruction and curriculum staff 
•A Test vendor 
Outside experts 
•A Special education teachers 
•A General education teachers 
Content specialists 
School psychologists/counselors 
•A School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

The state did not report a standard-setting 
methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Utah 
proficiency levels for 
alternate assessment 


Advanced 


Level 4: Substantial 


Proficient 


Level 3: Sufficient 


Basic 


Level 2: Partial 




Level 1: Minimal 



Alternate Achievement Descriptors 

Utah developed 

•A The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Sufficient descriptor in reading/ 
language arts is as follows: 

• A student scoring at this level is proficient 
on measured standards and objectives 
of the curriculum in this subject. The 
student’s performance indicates sufficient 
understanding and application of key 
curriculum concepts. 

Alternate Achievement Cut Scores 

Utah developed 

Unique cut scores for each grade 
Unique cut scores for grade spans 
One set of cut scores for all students 

•A Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at Level 4: Substantial 
and Level 3: Sufficient were proficient for purposes 
of the No Child Left Behind Act, whereas students 
who scored at Level 2: Partial and Level 1: Minimal 
were not proficient. 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

National Study on Alternate Assessments: State Profiles 



278 




Utah 



Technical Quality 



Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 








✓ 


State assessment staff 








✓ 


State instruction and curriculum staff 










Test vendor 


✓ 






✓ 


Outside experts 




✓ 


✓ 




Special education teachers 




✓ 


✓ 


✓ 


General education teachers 










Content specialists 


✓ 








School psychologists/counselors 










School/district/state administrators 










Parents 




✓ 




✓ 



Validity 




Reliability 




Type of validity 


Types of evidence 


Type of reliability 


Types of evidence 


Scoring and reporting 




✓ 


Variability across groups 


NCLB group 


structures consistent with 
the subdomain 






Internal consistency of item 


statistical analyses 


Relation to internal and 






responses 




external variables 




✓ 


Interrater consistency in 


Statistical analysis 


■/ Purposes of the assessment 


Not available for 




scoring 


conducted as part of 




examination 






actual scoring 


■/ Intended and unintended 


Anecdotal or 




Standard errors of 




consequences 


committee process 




measurement 




■/ Measurement of construct 


Anecdotal or 








relevance 

Grade-level equating 


committee process 


Other Types of Technical Quality 


Extended Content Standards 




Type of technical quality 


Types of evidence 






Alignment: 




The state developed extended content standards 
for students who were eligible for the alternate 


✓ 


Initial 


Formal study 


assessment. These extended content standards 


✓ 


Ongoing/monitoring 


External alignment 


were expressed by 






Fairness 


study 


v General link to state content standards 






Grade or grade span 




✓ 


Implementation processes 


Administration manual/ 






guide, monitoring 



Expanded benchmarks 
Alternate indicators or tasks 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Utah 



Eligibility and Administration 



Eligibility 

Utah provided the following guidelines to be used 
in determining when a child’s significant cognitive 
disability justified an alternate assessment based 
on alternate achievement standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. 

Assessment Content Selection 

State determined 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

•A Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

•A No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

•A State required standardized tasks/test items/ 
rating scales 

State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

Combination of work samples and other 
evidence 

•A No student work samples 

Assessment was administered 
•A As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

•A Teacher decided content to be assessed 
Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Utah 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

One scorer 

• Who scored? 

Student’s classroom teacher 

• How were scoring conflicts resolved? 

A third person adjudicated 
A third rater scored the assessment 

Y One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 

Y Ability to generalize across settings 

Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 

Y Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Y Work with nondisabled peers 

Y Appropriate human and technological 
supports 

None of the above 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 
Y' Scores 

Y Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 
■S Scores 

Y Standard/strand breakouts 
Indicator/benchmark breakouts 

Y Performance/achievement level descriptors 
Sample test items 



Training 





Assessment 


Assessment 


r 


\ 




administrators 


scorers 


State department of education 




Non-face-to-face 

training 

Face-to-face 

training 


✓ 


✓ 


website 

www.usoe.k12.ut.us/ 

V 


) 



Mandatory 


✓ 


✓ 


training and/or 






certification 







Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Utah 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated 

Reading/Language Arts 

Grades 




Percent 

□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



in state accountability assessments 

Mathematics 




Percent 

□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts 




Percent 



Mathematics 



Grades 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.92 


0.87 


6-8 


0.88 


0.82 


9-12 


0.70 


0.61 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. Data are unreliable for all 
students including students with lEPs in grades 9-12 who scored proficient or above on state accountability assessments in 
mathematics. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 




Vermont 



Overview of the Portfolio Assessment of Alternate Grade Expectations (PAAGE) 
( 2006 - 07 ) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Portfolio Assessment of 
Alternate Grade Expectations (PAAGE) were to 

V Evaluate programs 

Guide classroom instruction 

V Measure student progress/performance toward state standards 
Assess student access to content standards/general curriculum 
Assess individual students’ strengths/weaknesses 
Document academic achievement 

V Measure student progress toward IEP goals 



Alternate Assessment 

Portfolio Assessment of Alternate Grade 
Expectations (PAAGE) 

Alternate Assessment Approaches 

Rating scale/checklist 

V Portfolio/body of evidence 
Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

V Reading/language arts 

V Mathematics 
Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

V More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
3 of 4 standards 
Mathematics 

3 of 3 standards 

Who was involved in developing the alternate 
assessment? 

Assessment company 

V Research company/university/independent 
researcher 

Technical assistance provider 

V State personnel 
Parents 
Stakeholders 

Who administered the alternate assessment? 

V The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Student’s classroom teacher 

V School- or district-based educator 
State or state-contracted scorer 
Machine scored 
Paraprofessional 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Vermont 



Alternate Achievement Standards 

Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 

State instruction and curriculum staff 
Test vendor 
•A Outside experts 
•A Special education teachers 
•A General education teachers 
•A Content specialists 

School psychologists/counselors 
•A School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Dominant Profile Method standard-setting 
methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Vermont 
proficiency levels for 
alternate assessment 


Advanced 


Meets Expectations With 




Distinction 


Proficient 


Meets Expectations 


Basic 


Partially Meets Expectations 




Substantially Below Expectations 



Students who performed at the Meets Expectations 
With Distinction and Meets Expectations levels 
were proficient for purposes of the No Child Left 
Behind Act, whereas students who scored at the 
Partially Meets Expectations and Substantially 
Below Expectations levels were not proficient. 



Alternate Achievement Descriptors 

Vermont developed 

•A The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Meets Expectations descriptor in 

reading/language arts is as follows: 

• The student’s portfolio provides clear and 
sufficient evidence of effective program 
elements and a satisfactory degree of 
student success. Scores at this level are 
commonly associated with programs that 
have the following elements: The Goals 
and Objectives are aligned with the AGE, 
are measurable/observable, and reflect 
challenging, high priority, student-centered 
learning targets; The Program is based on 
effective, evidence-based methods, materials 
and strategies; Planning Team includes 
parents, teachers, educators and the student, 
and works collaboratively to develop, 
monitor and improve the program; Data 
collection is frequent and is used to monitor 
program success; and student progress on 
all targeted Alternate Grade Expectations 
reflects appropriate levels of skill acquisition, 
fluency, maintenance and generalization. 
Overall, these elements work together 
effectively, resulting in student progress and 
success that meets expectations. 

Alternate Achievement Cut Scores 

Vermont developed 

Unique cut scores for each grade 
Unique cut scores for grade spans 
One set of cut scores for all students 

•A Other approaches (e.g., applying a rubric to 
determine proficiency level) 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Vermont 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 




✓ 






State assessment staff 




✓ 






State instruction and curriculum staff 






✓ 




Test vendor 










Outside experts 


✓ 


✓ 


✓ 




Special education teachers 




✓ 


✓ 


✓ 


General education teachers 




✓ 


✓ 


✓ 


Content specialists 

School psychologists/counselors 

School/district/state administrators 






✓ 


✓ 


Parents 






✓ 


✓ 



Validity 



Reliability 



Type of validity Types of evidence 





Scoring and reporting 
structures consistent with 
the subdomain 






Relation to internal and 
external variables 


Other type of analysis 


✓ 


Purposes of the assessment 


Survey 


✓ 


Intended and unintended 
consequences 


Public reports 


✓ 


Measurement of construct 
relevance 

Grade-level equating 


Construct analyses 



Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

General link to state content standards 

V Grade or grade span 

V Expanded benchmarks 
Alternate indicators or tasks 



Type of reliability Types of evidence 



Variability across groups 

Internal consistency of item 
responses 

■/ Interrater consistency in 
scoring 

Standard errors of 
measurement 


Statistical analysis 
conducted as part of 
training and actual 
scoring 


Other Types of Technical Quality 


Type of technical quality 


Types of evidence 


Alignment: 
✓ Initial 


Formal study 


Ongoing/monitoring 

Fairness 

■/ Implementation processes 


Anecdotal or 




committee process 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Vermont 



Eligibility and Administration 



Eligibility 

Vermont provided the following guidelines to be 
used in determining when a child’s significant 
cognitive disability justified an alternate 
assessment based on alternate achievement 
standards 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. 

Assessment Content Selection 

State determined 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

•A Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

State provided instructions on types and 
amounts of evidence to be collected 

•A Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
•A Student work samples only 

Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 
•A As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

•A Teacher decided content to be assessed 
•A Teacher selected materials 

Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

Teacher interpreted/recorded student 
responses or scores 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Vermont 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

Two scorers 

• Who scored? 

School- or district-based educator 

• How were scoring conflicts resolved? 

V A third person adjudicated 

A third rater scored the assessment 

One person scored, or scores were 
combined 

Elements of student performance used in scoring 

V Accuracy of student response 

V Ability to generalize across settings 

V Amount of independence 

V Amount of progress 

Environmental elements used in scoring 

V Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

V Appropriate human and technological 
supports 

None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 


✓ 


✓ 



Individual Student Reports 

Individual student reports were given to 

V Parents 

V Schools and teachers 

Individual student results were expressed 
in terms of 

V State’s achievement standards 
Scores 

V Percentiles 

Interpretive guidance was developed for 
School-level administrators 

V Teachers 

V Parents 
Students 

Reports given to parents included the following 
information 

V Performance/achievement level 
Scores 

V Standard/strand breakouts 
Indicator/benchmark breakouts 
Performance/achievement level descriptors 
Sample test items 



f \ 

State department of education 
website 

http://education.vermont.gov/ 

V ) 



■/ Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Vermont 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



Mathematics 



Grades 



3-5 


PI 


89 


n 






op 

CD 


3 


91 


n 


9-12 


M 




20 


40 60 

Percent 


80 100 



Grades 



3 



□ 


90 


n 








j 


91 


m 


M 


0 20 


40 60 

Percent 


80 100 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts 




Percent 



Mathematics 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.74 


0.73 


6-8 


0.65 


0.58 


9-12 


M 


M 



M Missing. Information not provided by state. 

1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 




Virginia 



Overview of the Virginia Alternate Assessment Program (VAAP) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purpose of the 2006-07 Virginia Alternate 
Assessment Program (VAAP) was to 

Evaluate programs 
Guide classroom instruction 

V Measure student progress/performance toward state standards 
Assess student access to content standards/general curriculum 
Assess individual students’ strengths/weaknesses 
Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

Virginia Alternate Assessment Program (VAAP) 
Alternate Assessment Approaches 
Rating scale/checklist 

V Portfolio/body of evidence 
Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

V Reading/language arts 

V Mathematics 

V Science 

V Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

V More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
2 of 2 standards 
Mathematics 

5 of 5 standards 

Who was involved in developing the alternate 
assessment? 

V Assessment company 

V Research company/university/independent 
researcher 

V Technical assistance provider 

V State personnel 

V Parents 

V Stakeholders 

Who administered the alternate assessment? 

V The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Student’s classroom teacher 

V School- or district-based educator 

V State or state-contracted scorer 
Machine scored 
Paraprofessional 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

Virginia Alternate Assessment for the 2006-07 School Year 



289 




Virginia 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 
•A State instruction and curriculum staff 
Test vendor 
•A Outside experts 
•A Special education teachers 
•A General education teachers 
•A Content specialists 
•A School psychologists/counselors 
School/district/state administrators 
Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Body of Work standard-setting 
methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Virginia 
proficiency levels for 
alternate assessment 


Advanced 


Advanced 


Proficient 


Proficient 


Basic 


Needs Improvement 



Alternate Achievement Descriptors 

Virginia developed 

•A The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Proficient descriptor in reading/ 
language arts is as follows: 

• The student demonstrates individual 
achievement in many of the skills and 
knowledge within selected core academic 
standards that have been reduced in depth 
and complexity. Overall performance is 
considered reasonably sufficient. 

Alternate Achievement Cut Scores 

Virginia developed 

Unique cut scores for each grade 
•A Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Advanced and 
Proficient levels were proficient for purposes of the 
No Child Left Behind Act, whereas students who 
scored at the Needs Improvement level were not 
proficient. 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Virginia 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


✓ 






✓ 


State assessment staff 




✓ 




✓ 


State instruction and curriculum staff 


✓ 








Test vendor 


✓ 


✓ 




✓ 


Outside experts 


✓ 


✓ 


✓ 




Special education teachers 


✓ 


✓ 


✓ 


✓ 


General education teachers 


✓ 


✓ 


✓ 


✓ 


Content specialists 








✓ 


School psychologists/counselors 










School/district/state administrators 


✓ 


✓ 


✓ 




Parents 


✓ 









Validity 



Reliability 



Type of validity Types of evidence 

Scoring and reporting 
structures consistent with 
the subdomain 
Relation to internal and 
external variables 

Purposes of the assessment 

Intended and unintended 
consequences 

✓ Measurement of construct Anecdotal or 

relevance committee process 

Grade-level equating 



Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

V General link to state content standards 
Grade or grade span 
Expanded benchmarks 
Alternate indicators or tasks 



Type of reliability Types of evidence 



Variability across groups 

Internal consistency of item 
responses 

■/ Interrater consistency in 
scoring 

Standard errors of 
measurement 


Statistical analysis 
conducted as part of 
actual scoring 


Other Types of Technical Quality 


Type of technical quality 


Types of evidence 


Alignment: 
V Initial 


Anecdotal or 


✓ Ongoing/monitoring 


committee process 
Internal alignment 


✓ Fairness 


studies 
Bias review, not 


■/ Implementation processes 


conducted regularly 
Training, 




administration manual/ 
guide 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Virginia 



Eligibility and Administration 



Eligibility 

Virginia provided the following guidelines to be 
used in determining when a child’s significant 
cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

Parent signature was required 

•A Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

•A Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

•A No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
•A Student work samples only 

Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 
•A As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

•A Teacher decided content to be assessed 
•A Teacher selected materials 

Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

National Study on Alternate Assessments: State Profiles 



292 




Virginia 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 1 
One scorer 

• Who scored? 

School- or district-based educator, state or 
state-contracted scorer 

• How were scoring conflicts resolved? 

V A third person adjudicated 

A third rater scored the assessment 

One person scored, or scores were 
combined 

Elements of student performance used in scoring 

V Accuracy of student response 
Ability to generalize across settings 
Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

V None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 







Individual Student Reports 

Individual student reports were given to 

V Parents 

V Schools and teachers 

Individual student results were expressed 
in terms of 

V State’s achievement standards 

V Scores 
Percentiles 

Interpretive guidance was developed for 
School-level administrators 

V Teachers 

V Parents 
Students 

Reports given to parents included the following 
information 

V Performance/achievement level 
■S Scores 

V Standard/strand breakouts 
Indicator/benchmark breakouts 

V Performance/achievement level descriptors 
Sample test items 



f \ 

State department of education 
website 

www.doe.virginia.gov 

V 1 ) 



1 Portion of sample double scored. 

S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Virginia 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated 

Reading/Language Arts 

Grades 




Percent 

□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 1 

□ Exempt/absent 



in state accountability assessments 

Mathematics 




Percent 

□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 1 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 2 



Reading/Language Arts 



Grades 




Percent 



Mathematics 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


1.06 


1.04 


6-8 


1.16 


1.21 


9-12 


0.89 


0.29 



1 For this state this included an alternate assessment based on modified achievement standards. 

2 Of all students who received valid scores on the specific types of accountability assessment. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 




Washington 



Overview of the Washington Alternate Assessment System (WAAS) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Washington Alternate 
Assessment System (WAAS) were to 

Evaluate programs 
Guide classroom instruction 

Y Measure student progress/performance toward state standards 
Assess student access to content standards/general curriculum 

Y Assess individual students’ strengths/weaknesses 
Document academic achievement 

Y Measure student progress toward IEP goals 



Alternate Assessment 

Washington Alternate Assessment System 
(WAAS) 

Alternate Assessment Approaches 

Rating scale/checklist 

Y Portfolio/body of evidence 
Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 

Y Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
1 of 8 standards 
Mathematics 

1 of 5 standards 

Who was involved in developing the alternate 
assessment? 

Assessment company 

Y Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 

Y Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Student’s classroom teacher 
School- or district-based educator 

Y State or state-contracted scorer 
Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

Washington Alternate Assessment for the 2006-07 School Year 



295 




Washington 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 

State instruction and curriculum staff 
Test vendor 
•A Outside experts 
•A Special education teachers 
•A General education teachers 
•A Content specialists 
•A School psychologists/counselors 
•A School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Bookmark or Item Mapping standard- 
setting methodology. 



Alternate Achievement Descriptors 

Washington developed 

•A The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Meets Standard descriptor in 

reading/language arts is as follows: 

• Evidence in the portfolio shows that the 
student meets or exceeds her/his goal for 
one academic skill and approaches her/his 
goal for the second academic skill. The 
evidence also shows that the student applies 
both academic skills in two or more contexts; 
therefore, achievement shown in the portfolio 
is a mostly reliable demonstration of the skills 
linked to the targeted GLEs. 

Alternate Achievement Cut Scores 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Washington 
proficiency levels for 
alternate assessment 


Advanced 


Exceeds Standard 


Proficient 


Meets Standard 


Basic 


Approaches Standard 




Well Below Standard 



Washington developed 

Unique cut scores for each grade 
•A Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Exceeds Standard 
and Meets Standard levels were proficient for 
purposes of the No Child Left Behind Act, whereas 
students who scored at the Approaches Standard 
and Well Below Standard levels were not proficient 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Washington 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 








✓ 


State assessment staff 


✓ 




✓ 


✓ 


State instruction and curriculum staff 










Test vendor 


✓ 


✓ 


✓ 


✓ 


Outside experts 


✓ 


✓ 


✓ 


✓ 


Special education teachers 






✓ 


✓ 


General education teachers 






✓ 


✓ 


Content specialists 






✓ 


✓ 


School psychologists/counselors 










School/district/state administrators 






✓ 


✓ 


Parents 






✓ 


✓ 



Validity 



Type of validity Types of evidence 



✓ 


Scoring and reporting 
structures consistent with 
the subdomain 


Not available for 
examination 


✓ 


Relation to internal and 
external variables 


Correlational study, 
other type of analysis 


✓ 


Purposes of the assessment 


Not available for 
examination 




Intended and unintended 






consequences 




✓ 


Measurement of construct 
relevance 

Grade-level equating 


Construct analyses 



Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 

Y Grade or grade span 
Expanded benchmarks 
Alternate indicators or tasks 



Reliability 


Type of reliability 


Types of evidence 


■/ Variability across groups 


Not available for 
examination 


Internal consistency of item 
responses 




•S Interrater consistency in 
scoring 


Statistical analysis 
conducted as part of 
actual scoring 


Standard errors of 
measurement 





Other Types of Technical Quality 



Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Formal study 


✓ 


Ongoing/monitoring 


External alignment 






study 


✓ 


Fairness 


Bias review, not 






conducted regularly 


✓ 


Implementation processes 


Monitoring 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Washington 



Eligibility and Administration 



Eligibility 

Washington provided the following guidelines 
to be used in determining when a child’s 
significant cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

Parent signature was required 

•A Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

•A Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

•A No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
•A Student work samples only 

Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 
•A As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

•A Teacher decided content to be assessed 
•A Teacher selected materials 

Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Washington 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored ?' 

One scorer 

• Who scored? 

State or state-contracted scorer 

• How were scoring conflicts resolved? 

Y A third person adjudicated 

A third rater scored the assessment 

One person scored, or scores were 
combined 

Elements of student performance used in scoring 
Accuracy of student response 
•Y Ability to generalize across settings 
Amount of independence 

Y Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

Y None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 




Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 




✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 

Y Scores 
Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 

Y Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 

Y Scores 

Standard/strand breakouts 
Indicator/benchmark breakouts 

Y Performance/achievement level descriptors 
Sample test items 



( ^ 

State department of education 

website 

www.sbe.wa.gov/ 

V ) 



1 Portion of sample double scored. 

V Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

Washington Alternate Assessment for the 2006-07 School Year 



299 



Washington 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 



3-5 

6-8 

9-12 



_ 92 _ 



92 



83 



0 20 



HH 



40 60 

Percent 



80 100 



Grades 



3-5 

6-8 

9-12 



Mathematics 



93 



92 



84 



0 20 



— i 1 — 

40 60 

Percent 



hjd 



80 100 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Grades 



3-5 



6-8 



9-12 




0 20 40 60 80 100 



Percent 



Mathematics 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.67 


0.68 


6-8 


0.48 


0.54 


9-12 


0.56 


0.53 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 




West Virginia 



Overview of the Alternate Performance Task Assessment (APTA) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 Alternate Performance 
Task Assessment (APTA) were to 

V Evaluate programs 

V Guide classroom instruction 

V Measure student progress/performance toward state standards 
Assess student access to content standards/general curriculum 
Assess individual students’ strengths/weaknesses 

V Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

Alternate Performance Task Assessment (APTA) 
Alternate Assessment Approaches 
Rating scale/checklist 
Portfolio/body of evidence 
Performance task/events 

Y Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 
Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

Y One day to 2 weeks 

More than 2 weeks to 1 month 
More than 1 month to 2 months 
More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
Varied by grade level 1 
Mathematics 

Varied by grade level 2 

Who was involved in developing the alternate 
assessment? 

Y Assessment company 

Research company/university/independent 
researcher 

Y Technical assistance provider 

Y State personnel 

Y Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

Y A certified educator who was not the student’s 
teacher 

Y Paraprofessional 

Who scored the alternate assessment? 

Student’s classroom teacher 
School- or district-based educator 

Y State or state-contracted scorer 
Machine scored 
Paraprofessional 

1 The number of general content standards was 3. 

2 The number of general content standards was 5. 



V Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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West Virginia 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 

State instruction and curriculum staff 
•A Test vendor 
Outside experts 
•A Special education teachers 
•A General education teachers 
•A Content specialists 

School psychologists/counselors 
•A School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Performance Profile Method and 
the Contrasting Groups standard-setting 
methodologies. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding West Virginia 
proficiency levels for 
alternate assessment 


Advanced 


Above Mastery 


Proficient 


Mastery 


Basic 


Partial Mastery 




Novice 



Students who performed at the Above Mastery and 
Mastery levels were proficient for purposes of the 
No Child Left Behind Act, whereas students who 
scored at the Partial Mastery and Novice levels 
were not proficient. 



Alternate Achievement Descriptors 

West Virginia developed 

The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

•A Descriptors that were unique for each grade 
tested 

An example of the Mastery descriptor in reading/ 
language arts is as follows: 

• The student demonstrates fundamental 
knowledge by showing performance that 
meets the extended standards in reading, 
writing, listening, speaking, and viewing. 

The student will perform the following 
tasks without assistance: determine word 
meaning through a variety of strategies; 
comprehend meaning from literary materials 
to make predictions and answer questions; 
apply information from a variety of sources; 
produce writing for practical uses with correct 
punctuation and capitalization; listen in order 
to communicate effectively in different ways 
and for different purposes. 

Alternate Achievement Cut Scores 

West Virginia developed 
•A Unique cut scores for each grade 
Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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West Virginia 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


✓ 


✓ 


✓ 


✓ 


State assessment staff 


✓ 


✓ 


✓ 


✓ 


State instruction and curriculum staff 










Test vendor 


✓ 


✓ 


✓ 


✓ 


Outside experts 


✓ 




✓ 


✓ 


Special education teachers 


✓ 


✓ 


✓ 


✓ 


General education teachers 


✓ 




✓ 




Content specialists 


✓ 


✓ 


✓ 


✓ 


School psychologists/counselors 










School/district/state administrators 


✓ 


✓ 


✓ 


✓ 


Parents 


✓ 






✓ 



Validity 



Type of validity Types of evidence 





Scoring and reporting 
structures consistent with 
the subdomain 




✓ 


Relation to internal and 


Correlational study, 




external variables 


other type of analysis 


✓ 


Purposes of the assessment 


Not available for 
examination 


✓ 


Intended and unintended 


Not available for 




consequences 


examination 


✓ 


Measurement of construct 


Statistical analyses, 




relevance 

Grade-level equating 


construct analyses 



Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

V General link to state content standards 

V Grade or grade span 

V Expanded benchmarks 

V Alternate indicators or tasks 



Reliability 



Type of reliability 


Types of evidence 


✓ 


Variability across groups 

Internal consistency of item 
responses 


NCLB group 
statistical analyses 


✓ 


Interrater consistency in 


Statistical analysis 




scoring 


conducted as part of 
actual scoring 


✓ 


Standard errors of 
measurement 


Statistical analysis 



Other Types of Technical Quality 



Type of technical quality Types of evidence 

Alignment: 



✓ 


Initial 


Formal study 


✓ 


Ongoing/monitoring 


External and other 






alignment studies 


✓ 


Fairness 


Regularly scheduled 






bias review by experts 


✓ 


Implementation processes 


Administration manual/ 



guide, monitoring 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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West Virginia 



Eligibility and Administration 



Eligibility 

West Virginia provided the following guidelines 
to be used in determining when a child’s 
significant cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
Required modified instruction 

Required extensive support for skill 
generalization 

•A Required modified curriculum 
Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

Parent signature was required 

•A Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

•A All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

•A A certified educator who was not the 
student’s teacher 

•A Paraprofessional 

Procedures followed in gathering performance 
evidence included 

•A State required standardized tasks/test items/ 
rating scales 

State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

Combination of work samples and other 
evidence 

•A No student work samples 

Assessment was administered 

As part of day-to-day student instruction 
•A Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

Teacher decided content to be assessed 
Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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West Virginia 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

Two scorers 

• Who scored? 

State or state-contracted scorer 

• How were scoring conflicts resolved? 

A third person adjudicated 
V A third rater scored the assessment 

One person scored, or scores were 
combined 

Elements of student performance used in scoring 

V Accuracy of student response 
Ability to generalize across settings 

V Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

V None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 




Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 


✓ 


✓ 



Individual Student Reports 

Individual student reports were given to 

V Parents 

V Schools and teachers 

Individual student results were expressed 
in terms of 

V State’s achievement standards 

V Scores 
Percentiles 

Interpretive guidance was developed for 

V School-level administrators 

V Teachers 

V Parents 
Students 

Reports given to parents included the following 
information 

V Performance/achievement level 
■S Scores 

V Standard/strand breakouts 
Indicator/benchmark breakouts 

V Performance/achievement level descriptors 
Sample test items 



( ^ 

State department of education 

website 

http ://wvd e . state . wv. u s/oaa 

V ) 



■/ Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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West Virginia 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated 

Reading/Language Arts 

i^raaes 




Percent 

□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



in state accountability assessments 

Mathematics 




Percent 

□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts 




Percent 



Mathematics 



Grades 

3-5 





n 



0 20 40 60 80 100 



Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.89 


0.98 


6-8 


0.88 


0.84 


9-12 


0.79 


0.77 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 



Wisconsin 



Overview of the Wisconsin Alternate Assessment for Students with Disabilities 
(WAA-SWD) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

The purpose of the 2006-07 Wisconsin Alternate Assessment for Students with Disabilities (WAA-SWD) 
was to meet accountability requirements. 



Alternate Assessment 

Wisconsin Alternate Assessment for Students with 
Disabilities (WAA-SWD) 

Alternate Assessment Approaches 

Y Rating scale/checklist 

Y Portfolio/body of evidence 
Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 

Y Science 

Y Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

Y More than 1 month to 2 months 
More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
4 of 6 standards 
Mathematics 

6 of 6 standards 

Who was involved in developing the alternate 
assessment? 

Assessment company 

Y Research company/university/independent 
researcher 

Technical assistance provider 

Y State personnel 
Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

Y A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Y Student’s classroom teacher 
School- or district-based educator 
State or state-contracted scorer 
Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Wisconsin 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

•A State special education staff 
•A State assessment staff 

State instruction and curriculum staff 
Test vendor 
•A Outside experts 
•A Special education teachers 
•A General education teachers 
■A Content specialists 

School psychologists/counselors 
School/district/state administrators 
•A Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Judgmental Policy Capturing standard- 
setting methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Wisconsin 
proficiency levels for 
alternate assessment 


Advanced 


Prerequisite Skilled Advanced 


Proficient 


Prerequisite Skill Proficient 


Basic 


Prerequisite Skill Basic 




Prerequisite Skill Minimal 



Alternate Achievement Descriptors 

Wisconsin developed 

The same descriptors that applied to all 
grades tested 

•A The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the Prerequisite Skill Proficient 
descriptor in reading/language arts is as follows: 

• Student demonstrates the ability to decode 
and comprehend text with moderate support. 
Student’s understanding of basic concepts 
and performance of most reading skills are 
inconsistent, but typically at the developing or 
developed level across two or more settings. 

Alternate Achievement Cut Scores 

Wisconsin developed 

Unique cut scores for each grade 
•A Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Prerequisite Skilled 
Advanced and Prerequisite Skill Proficient levels 
were proficient for purposes of the No Child Left 
Behind Act, whereas students who scored at the 
Prerequisite Skill Basic and Prerequisite Skill 
Minimal levels were not proficient. 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Wisconsin 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 1 


State special education staff 


✓ 


✓ 


✓ 




State assessment staff 

State instruction and curriculum staff 

Test vendor 


✓ 


✓ 


✓ 




Outside experts 


✓ 


✓ 


✓ 




Special education teachers 
General education teachers 
Content specialists 
School psychologists/counselors 
School/district/state administrators 
Parents 


✓ 


✓ 


✓ 

✓ 





1 State did not address fairness. 



Validity 



Type of validity 


Types of evidence 




Scoring and reporting 
structures consistent with 
the subdomain 




✓ 


Relation to internal and 
external variables 


Correlational analysis 
using external 
measures, other type 
of analysis 


✓ 


Purposes of the assessment 


Anecdotal or 
committee process 


✓ 


Intended and unintended 
consequences 


Survey, post hoc data 
collection/analysis 


✓ 


Measurement of construct 
relevance 

Grade-level equating 


Statistical analyses 



Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

•Y General link to state content standards 
•Y Grade or grade span 
Y Expanded benchmarks 
Alternate indicators or tasks 



Reliability 



Type of reliability Types of evidence 





Variability across groups 




✓ 


Internal consistency of item 


Formal study or expert 




responses 


panel review 


✓ 


Interrater consistency in 


Statistical analysis 




scoring 


conducted as part of 
actual scoring 


✓ 


Standard errors of 
measurement 


Statistical analysis 



Other Types of Technical Quality 



Type of technical quality Types of evidence 

Alignment: 

V Initial Formal study 

•/ Ongoing/monitoring External alignment 

study 

Fairness 

Implementation processes 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Wisconsin 



Eligibility and Administration 



Eligibility 

Wisconsin provided the following guidelines to 
be used in determining when a child’s significant 
cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

Had a severe cognitive disability 
•A Required modified instruction 

•A Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. These procedures 
included 

Parent signature was required 

•A Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined 

•A All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

An observer/monitor who was present 

A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

•A No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

•A A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

•A Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

•A Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 

As part of day-to-day student instruction 
Separately from student’s daily work 

•A Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

Teacher decided content to be assessed 
•A Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Wisconsin 



Scoring and Reporting 
Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

Two scorers 

• Who scored? 

Student’s classroom teacher 

• How were scoring conflicts resolved? 

Y A third person adjudicated 

A third rater scored the assessment 

One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 

•Y Ability to generalize across settings 

Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 

Y Instruction in multiple settings 

Y Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Y Appropriate human and technological 
supports 

None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 







Individual Student Reports 

Individual student reports were given to 

Y Parents 

Schools and teachers 

Individual student results were expressed 
in terms of 

Y State’s achievement standards 

Y Scores 

Y Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 

Y Scores 

Standard/strand breakouts 
Indicator/benchmark breakouts 
Performance/achievement level descriptors 
Sample test items 



( \ 

State department of education 

website 

http://dpi.wi.gov/oea/wkce.html 

V ) 



■/ Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Wisconsin 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated 

Reading/Language Arts 

i^raaes 




0 20 40 60 80 100 

Percent 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



in state accountability assessments 

Mathematics 

90 11 

90 I) 

3 

89 II 

T 1 1 1 

0 20 40 60 80 100 

Percent 

□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Grades 

3-5 

6-8 

9-12 



Percentage of students who scored proficient or above on state accountability 
assessments in 1 



Reading/Language Arts 




Percent 



Mathematics 




Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


1.03 


0.87 


6-8 


0.88 


0.84 


9-12 


0.73 


0.69 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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National Study on Alternate Assessments: State Profiles 




Wyoming 



Overview of The Proficiency Assessments for Wyoming Students-Alternate 
Assessment (PAWS-ALT) (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 The Proficiency 
Assessments for Wyoming Students-Alternate Assessment (PAWS-ALT) were to 

Evaluate programs 

Y Guide classroom instruction 

Y Measure student progress/performance toward state standards 
Assess student access to content standards/general curriculum 

Y Assess individual students’ strengths/weaknesses 

Y Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

The Proficiency Assessments for Wyoming 

Students-Alternate Assessment (PAWS-ALT) 

Alternate Assessment Approaches 

Y Rating scale/checklist 

Y Portfolio/body of evidence 

Y Performance task/events 

Multiple choice/constructed response 
Content Areas Assessed 

Y Reading/language arts 

Y Mathematics 
Science 
Social studies 
Functional skills 

Grades Assessed 

✓ ✓✓✓✓✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration 

One day to 2 weeks 

More than 2 weeks to 1 month 

More than 1 month to 2 months 

Y More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts 
2 of 3 standards 
Mathematics 

5 of 5 standards 

Who was involved in developing the alternate 
assessment? 

Y Assessment company 

Research company/university/independent 
researcher 

Y Technical assistance provider 

Y State personnel 
Parents 

Y Stakeholders 

Who administered the alternate assessment? 

Y The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? 

Y Student’s classroom teacher 
School- or district-based educator 

Y State or state-contracted scorer 
Machine scored 
Paraprofessional 



Y Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Wyoming 



Alternate Achievement Standards 



Who was involved in creating alternate 
achievement standards? 

State special education staff 
•A State assessment staff 
•A State instruction and curriculum staff 
•A Test vendor 
•A Outside experts 
•A Special education teachers 
•A General education teachers 
■A Content specialists 

School psychologists/counselors 
School/district/state administrators 
Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the Body of Work standard-setting 
methodology. 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding Wyoming 
proficiency levels for 
alternate assessment 


Advanced 


Advanced 


Proficient 


Proficient 


Basic 


Basic 




Below Basic 



Alternate Achievement Descriptors 

Wyoming developed 

The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

•A Descriptors that were unique for each grade 
tested 

An example of the Proficient descriptor in reading/ 
language arts is as follows: 

• Students performing at a proficient level meet 
the standard of understanding literary and 
informational texts and perform in familiar 
settings. 

Alternate Achievement Cut Scores 

Wyoming developed 
•A Unique cut scores for each grade 
Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



Students who performed at the Advanced and 
Proficient levels were proficient for purposes of the 
No Child Left Behind Act, whereas students who 
scored at the Basic and Below Basic levels were 
not proficient. 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Wyoming 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 






✓ 


✓ 


State assessment staff 


✓ 


✓ 


✓ 


✓ 


State instruction and curriculum staff 


✓ 


✓ 


✓ 


✓ 


Test vendor 


✓ 


✓ 


✓ 


✓ 


Outside experts 


✓ 


✓ 


✓ 


✓ 


Special education teachers 






✓ 


✓ 


General education teachers 






✓ 


✓ 


Content specialists 






✓ 


✓ 


School psychologists/counselors 






✓ 


✓ 


School/district/state administrators 






✓ 


✓ 


Parents 






✓ 


✓ 



Validity 



Type of validity 


Types of evidence 




Scoring and reporting 
structures consistent with 
the subdomain 




✓ 


Relation to internal and 
external variables 


Correlational study 


✓ 


Purposes of the assessment 


Anecdotal or 
committee process 


✓ 


Intended and unintended 
consequences 


Survey 


✓ 


Measurement of construct 


Statistical analyses, 




relevance 

Grade-level equating 


construct analyses 



Extended Content Standards 

The state developed extended content standards 
for students who were eligible for the alternate 
assessment. These extended content standards 
were expressed by 

Y General link to state content standards 

Y Grade or grade span 

Y Expanded benchmarks 

Y Alternate indicators or tasks 



Reliability 



Type of reliability 


Types of evidence 




Variability across groups 




✓ 


Internal consistency of item 
responses 


Formal study or expert 
panel review 


✓ 


Interrater consistency in 
scoring 


Statistical analysis 
conducted as part of 
actual scoring 


✓ 


Standard errors of 
measurement 


Statistical analysis 


Other Types of Technical Quality 


Type of technical quality 


Types of evidence 




Alignment: 




✓ 


Initial 


Formal study 


✓ 


Ongoing/monitoring 


External alignment 
study 


✓ 


Fairness 


Bias review, not 
conducted regularly 


✓ 


Implementation processes 


Training, 
administration manual/ 
guide, post hoc data 
collection/analysis 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

Wyoming Alternate Assessment for the 2006-07 School Year 



315 



Wyoming 



Eligibility and Administration 



Eligibility 

Wyoming provided the following guidelines to 
be used in determining when a child’s significant 
cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. 

•A Had a severe cognitive disability 
•A Required modified instruction 

Required extensive support for skill 
generalization 

•A Required modified curriculum 
•A Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. 

Assessment Content Selection 

State determined 

•A All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by 

•A An observer/monitor who was present 

•A A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by 
•A The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included 

•A State required standardized tasks/test items/ 
rating scales 

•A State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included 
Student work samples only 

•A Combination of work samples and other 
evidence 

No student work samples 

Assessment was administered 

As part of day-to-day student instruction 
Separately from student’s daily work 

•A Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements 

Teacher decided content to be assessed 
•A Teacher selected materials 

•A Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

•A Teacher interpreted/recorded student 
responses or scores 



v' Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Scoring and Reporting 
Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 1 
One scorer 

• Who scored? 

Student’s classroom teacher, state or state- 
contracted scorer 

• How were scoring conflicts resolved? 

A third person adjudicated 
A third rater scored the assessment 

Y One person scored, or scores were 
combined 

Elements of student performance used in scoring 

Y Accuracy of student response 

Y Ability to generalize across settings 

Y Amount of independence 
Amount of progress 

Environmental elements used in scoring 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

Y None of the above 



Training 





Assessment 

administrators 


Assessment 

scorers 


Non-face-to-face 

training 


✓ 


✓ 


Face-to-face 

training 


✓ 


✓ 


Mandatory 
training and/or 
certification 


✓ 


✓ 



Individual Student Reports 

Individual student reports were given to 

Y Parents 

Y Schools and teachers 

Individual student results were expressed 
in terms of 

Y State's achievement standards 

Y Scores 
Percentiles 

Interpretive guidance was developed for 

Y School-level administrators 

Y Teachers 

Y Parents 
Students 

Reports given to parents included the following 
information 

Y Performance/achievement level 

Y Scores 

Y Standard/strand breakouts 
Indicator/benchmark breakouts 

Y Performance/achievement level descriptors 
Sample test items 



f 'N 

State department of education 
website 

www.k12.wy.us/ 

V ) 



1 Portion of sample double scored. 

S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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Participation and Proficiency (2006-07) 



Ways that students with lEPs participated 

Reading/Language Arts 




Percent 



in state accountability assessments 

Mathematics 




Percent 



□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 1 



Reading/Language Arts 




Mathematics 




Percent 



Percent 



□ students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ Students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ Students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.72 


0.69 


6-8 


0.72 


0.77 


9-12 


0.68 


0.73 



1 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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Document Analyses and Verification 
Activities 

The primary source of data for the NSAA 
State Profiles and NSAA National Profile is 
the documentation that states submitted to the 
U.S. Department of Education’s (ED) Office 
of Elementary and Secondary Education in 
response to the Standards and Assessment 
Peer Review (Peer Review) process. The 
Peer Review process is an ongoing process to 
evaluate whether states’ assessment systems 
meet No Child Left Behind Act of 2001 
(NCLB) requirements. The Standards and 
Assessment Peer Review Guidance provided 
by the Office of Elementary and Secondary 
Education framed the data collection activities 
for the NSAA State and National Profiles, 
as recommended by a panel of experts. 

States’ submissions to the Peer Review 
included the following seven sections. Each 
section included elements that defined how 
states’ various assessments met established 
professional standards and ED requirements: 

• challenging academic content 
standards; 

• challenging academic achievement 
standards; 

• a single statewide system of annual 
high-quality assessments; 

• high technical quality; 

• alignment of academic content 
standards, academic achievement 
standards, and assessments; 

• inclusion of all students in the 
assessment system; and 

• assessment reporting. 

The Peer Review submissions had several 
advantages for the purposes of the NSAA. 
First, they provided a common framework 
to which all states responded. Second, the 
responses included much of the evidence 
in a single location. Third, the responses 



and evidence provided were likely to be 
reliable for the 2005-06 and 2006-07 school 
years. Fourth, the Peer Review sections and 
elements addressed issues related to states’ 
alternate assessment systems in light of the 
states’ overall assessment systems. Fifth, 
the submissions provided an opportunity 
to observe how states responded to issues 
raised by peer reviewers. 

The SRI study team and its partners used 
two data collection methods to investigate 
the status and direction of state alternate 
assessments for children with significant 
cognitive disabilities between summer 2006 
and fall 2007. First, the team reviewed in depth 
the state document submissions to the Peer 
Review process and information pertaining to 
the alternate assessments on state websites. 
Second, the study team conducted structured 
telephone interviews with knowledgeable 
informants in each of the 50 states and the 
District of Columbia. The purpose of these 
interviews was to verify findings from the 
document review and to obtain additional 
information about assessments that could not 
be gleaned from states’ submissions. 

To document its review and analysis of 
the state documents, SRI developed a data 
collection instrument and web-based database 
and data collection system for compiling 
states’ responses to the elements in the 
Standards and Assessment Peer Review 
Guidance, additional information gathered 
from state websites, and states’ responses 
to the subsequent interviews. The seven 
Peer Review components and corresponding 
elements provided in the guidance document 
became the basis for the data collection 
instrument. Instrument items were phrased in 
the form of a question similar to the phrasing 
in the Peer Review guidance. This phrasing 
provided a standard way for asking a state 
respondent to provide information that was 
not contained in the document review. A 
panel of experts reviewed the initial data 
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collection instrument items to ensure that 
the items accurately reflected the intent of 
the Peer Review elements. A few additional 
items (e.g., number of content standards 
assessed on the alternate assessment) were 
recommended by ED and the same panel of 
experts as important for documenting alternate 
assessment systems and were included 
in the instrument. The full instrument was 
administered to State Department of Education 
officials from four states as part of the piloting 
process. These individuals provided feedback 
on the clarity of the items and assisted NSAA 
in determining the feasibility of the document 
analysis and verification process and the 
procedures. State Department of Education 
officials did not suggest any changes to the 
items. They did, however, suggest approaches 
to facilitate data collection and reduce states’ 
burden and time commitment. It should be 
noted that it was not necessary to administer 
all instrument items to state respondents. 

State respondents were administered only 
items that were not completed based on the 
document review. 

During the initial data collection phase, 
the evolving nature of alternate assessments 
based on alternate achievement standards 
became evident. As states received feedback 
through the Peer Review process, the majority 
found it necessary to revise and, in some 
cases, to discontinue use of their alternate 
assessments administered in 2005-06. Thus, 
while the original plan had been to focus 
on the 2005-06 school year, as a result of 
changes taking place in states’ alternate 
assessment systems, the study team decided, 
in consultation with ED and a panel of experts, 
to collect data for both the 2005-06 and 
2006-07 school years. 

In June 2006, research team members 
were trained to use a systematic set of 
procedures for analyzing state documents 
and websites and for entering the data into 
the database. Two analysts were assigned to 



each state, with the intention of having two 
researchers reviewing each state’s extensive 
documentation and becoming highly familiar 
with the state’s information. Procedures for 
data collection and the role and responsibilities 
of each researcher were clearly defined, 
based on the following steps. 

Step 1. One researcher, identified as R1, 
reviewed the state’s submission narrative 
and the Peer Review Consensus notes, and 
conducted the initial web-based search for 
policy documents, state academic content 
standards (including “extended” academic 
content standards for students with significant 
cognitive disabilities), alternate assessment 
training manuals and technical manuals, 
and alternate assessment blueprints or test 
specifications that were available on state 
department of education websites. R1 entered 
the information into the database. When 
R1 had completed the review of electronic 
and online materials as fully as possible, 
he or she turned the findings over to the 
second assigned researcher (R2). During 
this “handover,” R1 briefed R2 on missing 
information and where this information might 
be located in the Peer Review submission 
materials housed at ED. 

Step 2. The R2 researcher then went 
to the ED headquarters building where the 
complete set of Peer Review submissions 
were stored to locate documents pertinent to 
the state’s alternate assessment. Examples 
of documents sought and reviewed included 
technical reports, results of internal and / 
or external validity and reliability studies, 

Board of Education minutes, notes from 
state Technical Advisory Committees, the 
minutes from sessions for setting alternate 
achievement standards, results of alignment 
studies, and state timelines for meeting Peer 
Review requirements. R2 then entered the 
findings of this review into the study database. 
Throughout, both R1 and R2 team members 
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reconfirmed the accuracy of their respective 
reviews and identified areas that needed 
further clarification or resolution by a third 
supervising researcher (R3). The information 
collected on each state was reviewed by R3 
to promote consistency in responses and to 
reconcile any differences between R1 and R2 
findings. 

Step 3. When the data collection for 
each state was completed, the data were 
downloaded into a data verification version of 
the NSAA State Data Summary that presented 
the information collected for each state’s 
alternate assessment system and provided a 
mechanism for states to verify the researchers’ 
findings. The verification instrument (see 
figure A-1) included check boxes that allowed 
states to indicate whether the information 
collected was 

• accurate and complete for 2005-06; or 

• not accurate and/or not complete for 
2005-06; and whether 

• information had changed for 2006-07. 

The verification instrument was piloted 
in four states during December 2006 and 
January 2007. On the basis of feedback from 
these states, and in consultation with ED, 
the NSAA State Data Summary verification 
process was further streamlined and includes 
the items presented in this report. 

In March 2007, the study team sent a 
letter to the state director of assessment and 
accountability and the state superintendent 
of public instruction in each state and the 
District of Columbia from the Commissioner 
of the National Center for Special Education 
Research. This letter described the purpose 
of the study, introduced SRI and its partners, 
and asked states to identify the persons most 
appropriate to review the NSAA State Data 
Summary and participate in a telephone 
interview. 



The NSAA State Data Summary was sent 
to each state between March 27 and May 8, 
2007, with detailed instructions to the state 
informant(s) on completing and returning the 
summary to SRI within 2 weeks. 

Step 4. In March 2007, the research team 
was trained on the procedures to be followed 
in conducting the telephone interviews with 
state administrators. A lead researcher and a 
support researcher were identified for each 
state. These researchers were usually the 
same individuals who had conducted the 
document analysis. SRI developed a website 
to record the results of the state interviews; 
the site included a call log, data entry screen 
for interviewing, and mechanism for combining 
notes taken during the interview by the two 
participating researchers. 

Step 5. When states returned their 
completed NSAA State Data Summary 
reviews (April to September 2007), a research 
team member entered states’ responses into 
the NSAA database. The lead researcher for 
each state arranged a convenient interview 
time with the state informant(s). The lead 
researcher asked each state to provide 
information only about items informants 
indicated were not accurate or not complete 
for 2005-06 and those for which information 
had changed from 2005-06 to 2006-07. 

During the interview, two researchers recorded 
interview responses and comments about 
the data into the NSAA database. At this 
time, the researchers also developed a list 
of documents not previously available to 
the study team that states agreed to send 
to NSAA as they became available. These 
documents included, for example, new training 
manuals and technical reports about the 
alternate assessments’ reliability, validity, or 
alignment with state standards for the 
2006-07 school year. 

Step 6. Following completion of the 
telephone interview, the two interviewers 
updated and edited the state information on 
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Figure A-1. Verification instrument (page 1) 

[STATE NAME] 
Alternate Assessment 
Data Summary 
May 17, 2007 

Overview 

Completed By: Phone: 



E-mail: 



1. (Ala) Assessment Title 




Document Analysis Response: 

[STATE DATA] 


Select if. Accurate and Complete for 2005-06 or Not Accurate and/or Not Complete for 2005-06 

Check if applicable: Information has changed for 2006-07 school year 





1 

Notes (for your convenience): 




2. (Alb) Assessment Developer 




Document Analysis Response: 

[STATE DATA] 



Select if, Accurate and Complete for 2005-06 or Not Accurate and/or Not Complete for 2005-06 

Check if applicable: Information has changed for 2006-07 school year 



Notes (for your convenience): 



3. (Ale) Content Area 
Document Analysis Response: 



Unknown Language arts 


Math 


Science Social studies 





Select if, Accurate and Complete for 2005-06 or Not Accurate and/or Not Complete for 2005-06 

Check if applicable: Information has changed for 2006-07 school year 
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the NSAA database to include data collected 
prior to the interview that had been verified 
as correct by the state informant(s), data from 
the interview, and data from any additional 
documents cited during the interview and 
subsequently provided by the state to NSAA 
for review. This process was completed for all 
states and the District of Columbia by the end 
of September 2007. 

Data Processing and Coding Activities 

The NSAA data collected through the 
document review and telephone interview 
process resulted in four types of data formats: 
yes/no items, multiple-choice items, closed- 
ended text items (such as the name of the 
assessment and number of content standards 
addressed for a specific subject), and open- 
ended response items. 

Coding of open-ended items. In 

September 2007, senior NSAA researchers 
met for 3 days to develop procedures for 
coding open-ended items. They used the 
following inductive analytic procedures for 
systematically analyzing qualitative data, as 
defined by Glaser and Strauss (1967) and 
Strauss and Corbin (1990). 

Step 1. For each open-ended item, 
the researchers worked in pairs to read 
and understand the state responses and 
to create initial coding categories. Each 
researcher in the pair then independently 
coded approximately 10 randomly selected 
state responses by reading line by line 
and assigning coding categories. The two 
researchers then discussed their proposed 
coding categories in detail, defining the 
reasoning behind each code and its definition 
and then reconciling differences and refining 
existing codes or adding others as needed. 
The researchers then independently coded 
another 10 items to test the proposed coding 
scheme. 



Step 2. Over a 2-week period, codes 
for all sections were revised and examined 
to prepare for a “Coding Institute.” State 
responses to most of the items could be 
differentiated into a relatively few easily coded 
categories. When necessary, redundant or 
overlapping items were collapsed and coded 
together. A small number of items elicited little 
or no information because the items applied 
to too few or no states’ alternate assessment 
systems. For example, no state had multiple 
alternate assessment forms, so coding 
categories were not developed for the items 
referring to multiple test forms. 

Step 3. The Coding Institute was a 
week-long meeting that included training 
and practice for researchers on using the 
codes, followed by coding the open-ended 
items for the 50 states and the District of 
Columbia. A pair of researchers coded each 
item, first coding the item independently, then 
comparing their codes, and finally reconciling 
any disagreements for a final code (see 
Interrater Reliability section below). In some 
instances, the data provided by the state 
were ambiguous and could not be coded. 
These items were further researched and 
then were subjected to the same process of 
having two researchers independently code 
them, followed by a comparison of codes and 
a reconciliation of any disagreements for a 
final code. Codes were recorded on hard-copy 
coding sheets. 

Step 4. The data from the hard-copy 
coding sheets were double entered (to ensure 
accuracy) into Excel files, from which a data 
file for each state was generated that included 
all items and their response codes for final 
review and verification by the lead researcher/ 
interviewer for each state. 

Step 5. Coding was verified for all 50 
states and the District of Columbia. The lead 
researcher/interviewer reviewed all responses 
for consistency, based on his or her knowledge 
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and understanding of the state. Updates and 
revisions were made to a few of the items, and 
those changes were documented in the final 
dataset. 

Processing of uncoded items. Data from 
the yes/no, multiple-choice, and closed-ended 
items did not require coding as described 
above, and were verified and recorded in 
spreadsheets. 

State Profile Data Verification 

State profiles were created from the 
collected data. After consultation with ED staff, 
a decision was made to focus the profiles 
only on the 2006-07 school year rather 
than both the 2005-06 and 2006-07 school 
years, for which data had been collected. The 
decision allowed the most up-to-date data to 
be reported clearly and without redundancy. 
Because the state profiles displayed data in 
a different format than states had previously 
reviewed, the state profile was sent to each 
state for a final review in May and June 2008, 
using the following procedure: 

Step 1. Each state profile was sent to 
the state assessment director or previous 
respondent for review. A cover letter explained 
that changes to the profile were possible 
only if they could be supported by state 
documentation. 

Step 2. Each state was contacted by 
phone and e-mail to discuss the profile. 
Information and documentation were collected 
for items that might be considered for update 
or revision. 

Step 3. The research team reviewed any 
information and documentation submitted 
by each state. A profile was updated to add 
missing information or correct inaccuracies if 
sufficient documentation was provided by the 
state. 



Interrater Reliability 

Open-Ended Data Coding Activities 

Two sets of comparisons were calculated 
for the interrater reliability of the open-ended 
items. The first consisted of comparisons 
between the codes for Rl’s and R2’s individual 
results from the Coding Institute (see step 3, 
p. A-7). The second set of comparisons 
calculated the interrater reliability of the 
reconciled final codes determined by R1 and 
R2 from the Coding Institute (step 3) and 
codes determined during the final verification 
by the lead researcher/interviewer (step 5). 
Each comparison was calculated by percent 
agreement and Cohen’s Kappa analyses. 

The percent agreement calculation used the 
number of agreements 1 divided by the sum of 
agreements and disagreements. The Cohen’s 
Kappa calculation used the number observed 
to agree minus the number expected to agree 
by chance divided by the number of items 
minus the number expected to agree. 

Percent agreement analyses. The 
following interrater reliability findings apply 
only to the 2006-07 school year because only 
those data are reported in the profiles. In the 
first set of comparisons, the overall interrater 
agreement level between the two coders 
was 92.3 percent. In addition, calculations 
were conducted on an item-by-item basis 
and by response category for each item. The 
interrater agreement by item ranged from 
84.1 percent to 99.1 percent. The interrater 
agreement by response category for each 
item ranged from 75.9 percent to 100 percent 
(figure A-2). 

1 Agreements and disagreements were determined 
by comparing the codes for each response 
category. Agreements were assigned the value of 
100; disagreements were assigned the value of 0. 
Averages of all agreements and disagreements for 
the 50 states and the District of Columbia for all the 
response categories across all the items are reported 
as the overall percentage of agreement for each year. 
Averages of all items and of response categories for 
each item are reported at the item level. 
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Figure A-2. Interrater reliability percentages for percent agreement and Cohen’s Kappa between coders 



Percent agreement between initial coders, 
by response category for 2006-07 



Cohen’s Kappa between initial coders, 
by response category for 2006-07 



Number of 
categories 




Percent agreement 



Number 
of categories 




Kappa 



Percent agreement between initial and verification coders, 
by response category for 2006-07 



Cohen’s Kappa between initial and verification coders, 
by response category for 2006-07 
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categories 
80 
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The second set of comparisons calculated 
the interrater reliability of the reconciled 
final codes determined by R1 and R2 from 
the Coding Institute (step 3) and codes 
determined during the final verification by 
the lead researcher/interviewer (step 5). The 
same approach in calculating the percentage 
of agreement between coders was used. 

The overall interrater agreement level was 

93.7 percent. In addition, as with the first 
set of comparisons, the interrater reliability 
was calculated for each item and for each 
response category. The interrater agreement 
ranged from 78.4 percent to 97.5 percent 
for the item-by-item analysis. The interrater 
agreement by response category ranged from 

55.8 percent (a single outlier) to 100 percent. 

Cohen’s Kappa analyses. The overall 
Cohen’s Kappa between the two coders was 
.80. The interrater agreement by item ranged 
from .59 to .96. The interrater agreement by 
response category for each item ranged from 
-.03 to 1 .0 (figure A-2). 

For the second set of comparisons, the 
reconciled final codes determined by R1 and 
R2 from the Coding Institute (step 3) and 
codes determined during the final verification 
by the lead researcher/interviewer (step 5), the 
overall interrater agreement level was .84. In 
addition, as with the first set of comparisons, 
the interrater reliability was calculated for each 
item and for each response category. The 
interrater agreements ranged from .58 to .95 
for the item-by-item analysis. The interrater 
reliability coefficients by response category 
ranged from 0 to 1.0. 

The low values for kappa tended to 
occur on items where the raters both agreed 
for almost all states but almost all of the 
ratings were in one of the two possible rating 
categories. For example, for item D5 the 
coders marked “student’s special education 
teacher” for most of the states and their 
agreement was high, but because the vast 



majority were marked as “student’s special 
education teacher,” few disagreements 
(relative to the total number of states) between 
coders 1 and 2 resulted in a low kappa value. 
We noted that when there was substantial 
imbalance in the percentage of observations 
in the two rating categories, kappa could 
give counterintuitive results. For example, 
suppose that each rater has a 90 percent 
chance of correctly rating a state but that all 
states belong to only one of the two rating 
categories. Then we would expect 81 percent 
agreement between the raters in the correct 
rating category, 1 percent agreement between 
the raters in the wrong rating category, and 
18 percent disagreement between the raters 
(split between the two off-diagonal cells in a 
2x2 rating table). This results in an expected 
kappa of .0 even though both raters have a 
90 percent correct rating ability. If, on the other 
hand, half of the states are in each of the two 
rating categories then, the expected kappa 
is .64. 

State Profile Verification Activities 

For the state profile verification activities, 
all items, both open-ended and uncoded 
(yes/no items, multiple-choice items, and 
closed-ended text items) were reviewed 
by states. One comparison was calculated 
for the interrater reliability during the state 
profile verification activities. The comparison 
calculated the interrater reliability between 
the final verification by the lead researcher/ 
interviewer (step 5) and the final review by 
the state of all items. Similar to the open- 
ended items, the comparison was calculated 
by percent agreement and Cohen’s Kappa 
analyses. Each analysis was examined 
separately for the open-ended and uncoded 
items. The yes/no items, multiple-choice items, 
and closed-ended text items were originally 
reported by the state during the interview 
phase. If the items were changed during the 
review process, that change was due to the 
state review. 
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Figure A-3. Interrater reliability percentages for percent agreement and Cohen’s Kappa between final 
coder and state review 



Percent agreement between final coder and state review, 
by response category for 2006-07 



Number of 
categories 




Percent agreement 



Percent agreement analyses. The overall 
interrater agreement level between the final 
verification by the researcher team and final 
review by the state for open-ended items 
was 90.7 percent. In addition, calculations 
were conducted on an item-by-item basis 
and by response category for each item. The 
interrater agreement by item ranged from 
75.0 percent to 100 percent. The interrater 
agreement by response category for each item 
ranged from 53.9 percent (a single outlier) to 
100 percent (figure A-3). 

For uncoded items, the overall interrater 
agreement level was 91 .8 percent (figure A-3). 
The interrater agreement by item ranged from 
84.8 percent to 100 percent. The interrater 
agreement by response category for each item 
ranged from 50.0 percent (a single outlier) to 
100 percent. 

Cohen’s Kappa analyses. The overall 
Cohen’s Kappa between the final verification 
by the researcher team and final review by 
the state was .80 (figure A-3). The interrater 
agreement by item ranged from .33 to 1 .0. The 



Cohen’s Kappa between final coder and state review, 
by response category for 2006-07 



Number 
of categories 




-.05 .05 .15 .25 .35 .45 .55 .65 .75 .85 .95 

Kappa 



interrater agreement by response category for 
each item ranged from -.03 to 1 .0. 

For uncoded items, the overall interrater 
agreement level was .84. The interrater 
agreement by item ranged from .69 to 1 .0. The 
interrater agreement by response category for 
each item ranged from 0 to 1.0 (figure A-3). 

Two items in the state profile (the number 
of content standards assessed by alternate 
assessment and the number of general 
content standards) were not included in the 
Cohen’s Kappa calculation because of the 
structure of the items. 

Similar to the open-ended coding activities, 
the low values for kappa tended to occur on 
items where the raters both agreed for almost 
all states but almost all of the ratings were 
in one of the two possible rating categories. 

We noted that when there was substantial 
imbalance in the percentage of observations 
in the two rating categories, kappa could give 
counterintuitive results. 
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Explanation of Item Codes 

Appendix B presents a description for each item, the response codes, and the procedures 
used to code the item. The primary source of data for these items is the documentation that 
states submitted to the U.S. Department of Education’s Office of Elementary and Secondary 
Education in response to the Standards and Assessment Peer Review (Peer Review) process 
and information pertaining to the alternate assessments on state websites. The study team also 
conducted structured interviews with knowledgeable informants in each of the 50 states and 
the District of Columbia. These interviews were conducted to collect data on items that could 
not be collected from administrative records and to verify that the information collected from 
the records was accurate. The items discussed during the interviews varied by state depending 
upon the information provided in the documentation provided by states. In other words, the 
interviews were not systematic surveys. Data were collected and coded according to procedures 
described in appendix A. 

The following template provides a framework for presenting data from each state using the data 
codes. 
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[State] 



Overview of the [ASSESSMENT TITLE (A1)] (2006-07) 

Purposes of the Alternate Assessment Based on Alternate Achievement Standards 

In addition to meeting accountability requirements, the purposes of the 2006-07 [ASSESSMENT TITLE 
(A1 )] were to [A2] 

Evaluate programs 
Guide classroom instruction 

Measure student progress/performance toward state standards 
Assess student access to content standards/general curriculum 
Assess individual students’ strengths/weaknesses 
Document academic achievement 
Measure student progress toward IEP goals 



Alternate Assessment 

[ASSESSMENT TITLE (A1)] 

Alternate Assessment Approaches [A3] 

Rating scale/checklist 
Portfolio/body of evidence 
Performance task/events 
Multiple choice/constructed response 
Content Areas Assessed [A4] 
Reading/language arts 
Mathematics 
Science 
Social studies 
Functional skills 
Grades Assessed [A5] 

✓ ✓ ✓ ✓ ✓ 

3 4 5 6 7 8 9 10 11 12 

Time Frame for Administration [A6] 

One day to 2 weeks 
More than 2 weeks to 1 month 
More than 1 month to 2 months 
More than 2 months 



Number of content standards assessed by the 
alternate assessment 

Reading/language arts [A7] 

A7b of A7a 
Mathematics [A8] 

A8b of A8a 

Who was involved in developing the alternate 
assessment? [A9] 

Assessment company 

Research company/university/independent 
researcher 

Technical assistance provider 

State personnel 

Parents 

Stakeholders 

Who administered the alternate assessment? 
[A10] 

The student’s special education teacher 

A certified educator who was not the student’s 
teacher 

Paraprofessional 

Who scored the alternate assessment? [All] 

Student’s classroom teacher 
School- or district-based educator 
State or state-contracted scorer 
Machine scored 
Paraprofessional 



V Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

[STATE] Alternate Assessment for the 2006-07 School Year 
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[State] 



Alternate Achievement Standards 

Who was involved in creating alternate 
achievement standards? [B1] 

State special education staff 

State assessment staff 

State instruction and curriculum staff 

Test vendor 

Outside experts 

Special education teachers 

General education teachers 

Content specialists 

School psychologists/counselors 

School/district/state administrators 

Parents 

Standard-Setting Methodologies 

Alternate achievement standards were developed 
using the [B2] 



Alternate Achievement Level Labels 



NCLB-required 
proficiency levels 


Corresponding [STATE] 
proficiency levels for 
alternate assessment 


Advanced 


[B3] 


Proficient 


[B3] 


Basic 


[B3] 



Students who performed at the [B3] and [B3] 
levels were proficient for purposes of the No Child 
Left Behind Act, whereas students who scored at 
the [B3] and [B3] levels were not proficient. 



Alternate Achievement Descriptors 

[STATE] developed [B4] 

The same descriptors that applied to all 
grades tested 

The same descriptors that applied to grade 
spans tested 

Descriptors that were unique for each grade 
tested 

An example of the [Proficient label] descriptor 
in reading/language arts [if grade/grade span 
provided: (grade 3)] is as follows* 

• [Text of sample descriptor] 



Alternate Achievement Cut Scores 

[STATE] developed [B5] 

Unique cut scores for each grade 
Unique cut scores for grade spans 
One set of cut scores for all students 

Other approaches (e.g., applying a rubric to 
determine proficiency level) 



*Descriptor posted on state website. 

v* Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 
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[State] 



Technical Quality 

Who was involved in reviewing the technical quality of the alternate assessment? 



Stakeholders 


Validity 


Reliability 


Alignment 


Fairness 


State special education staff 


[Cl] 


[C2] 


[C3] 


[C4] 


State assessment staff 


[Cl] 


[C2] 


[C3] 


[C4] 


State instruction and curriculum staff 


[Cl] 


[C2] 


[C3] 


[C4] 


Test vendor 


[Cl] 


[C2] 


[C3] 


[C4] 


Outside experts 


[Cl] 


[C2] 


[C3] 


[C4] 


Special education teachers 


[Cl] 


[C2] 


[C3] 


[C4] 


General education teachers 


[Cl] 


[C2] 


[C3] 


[C4] 


Content specialists 


[Cl] 


[C2] 


[C3] 


[C4] 


School psychologists/counselors 


[Cl] 


[C2] 


[C3] 


[C4] 


School/district/state administrators 


[Cl] 


[C2] 


[C3] 


[C4] 


Parents 


[Cl] 


[C2] 


[C3] 


[C4] 



Validity Reliability 



Type of validity Types of evidence 



[C5] 


Scoring and reporting 
structures consistent with 
the subdomain 


[C5] 


[C6] 


Relation to internal and 
external variables 


[C7] 


[C8] 


Purposes of the assessment 


[C9] 


[CIO] 


Intended and unintended 
consequences 


[C11] 


[Cl 2] 


Measurement of construct 
relevance 


[Cl 3] 


[Cl 4] 


Grade-level equating 


[Cl 4] 



Extended Content Standards 

The state [(Cl 5) developed/did not develop] 
extended content standards for students who 
were eligible for the alternate assessment. These 
extended content standards were expressed by 

[Cl 6] 

General link to state content standards 
Grade or grade span 
Expanded benchmarks 
Alternate indicators or tasks 



Type of reliability 


Types of evidence 


[Cl 7] 


Variability across groups 


[Cl 8] 


[Cl 9] 


Internal consistency of item 
responses 


[Cl 9] 


[C20] 


Interrater consistency in 
scoring 


[C21] 


[C22] 


Standard errors of 
measurement 


[C22] 


Other Types of Technical Quality 


Type of technical quality 


Types of evidence 




Alignment: 






Initial 


[C23] 




Ongoing/monitoring 


[C24] 


[C25] 


Fairness 


[C26] 


[C27] 


Implementation processes 


[C28] 



V Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

[STATE] Alternate Assessment for the 2006-07 School Year 
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[State] 



Eligibility and Administration 



Eligibility 

[STATE] provided the following guidelines to be 
used in determining when a child’s significant 
cognitive disability justified an alternate 
assessment based on alternate achievement 
standards. [D1] 

Had a severe cognitive disability 
Required modified instruction 

Required extensive support for skill 
generalization 

Required modified curriculum 
Not based on disability category 

Procedures were in place for informing parents 
when their child would be assessed on the 
alternate assessment based on alternate 
achievement standards. [D2] These procedures 
included 

Parent signature was required 

Parents were provided written materials 
about the alternate assessment 

Assessment Content Selection 

State determined [D3] 

All components - Academic content areas 
and strands, standards, benchmarks, and 
performance indicators/tasks 

Most components - Academic content areas 
and strands, and the IEP team determined 
performance indicators/tasks 

Some components - Only the academic 
content areas 

Monitoring and Verification 

Conducted by [D4] 

An observer/monitor who was present 

A local or school-level reviewer who 
confirmed proper administration of the 
assessment 

No independent verification process 



Administration 

Assessment conducted by [A10] 

The student’s special education teacher 

A certified educator who was not the 
student’s teacher 

Paraprofessional 

Procedures followed in gathering performance 
evidence included [D5] 

State required standardized tasks/test items/ 
rating scales 

State provided instructions on types and 
amounts of evidence to be collected 

Teacher/IEP team decided on types and 
amounts of evidence to be collected 

Student work submitted included [D6] 

Student work samples only 

Combination of work samples and other 
evidence 

No student work samples 
Assessment was administered [D7] 

As part of day-to-day student instruction 
Separately from student’s daily work 

Using a combination of day-to-day and on- 
demand approaches 

Based on teacher recollection of student 
performance 

Assessment required teacher judgment that 
included these elements [D8] 

Teacher decided content to be assessed 
Teacher selected materials 

Teacher made decisions about administering 
the assessment (e.g., timing, duration, 
support, scaffolding, test window) 

Teacher interpreted/recorded student 
responses or scores 



S Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

National Study on Alternate Assessments: State Profiles 
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[State] 



Scoring and Reporting 



Scoring 

The policy regarding the scoring of the alternate 
assessment included these elements: 

• How many people scored? 

[El] 

• Who scored? 

[All] 

• How were scoring conflicts resolved? [E2] 

A third person adjudicated 
A third rater scored the assessment 

One person scored, or scores were 
combined 

Elements of student performance used in scoring 

[E3] 

Accuracy of student response 
Ability to generalize across settings 
Amount of independence 
Amount of progress 

Environmental elements used in scoring [E4] 
Instruction in multiple settings 

Opportunities to plan, monitor, and evaluate 
work 

Work with nondisabled peers 

Appropriate human and technological 
supports 

None of the above 



Training 





Assessment 

administrator 


Assessment 

scorers 


Non-face-to-face 

training 


[E5] 


[E6] 


Face-to-face 

training 


[E5] 


[E6] 


Mandatory 
training and/or 
certification 


[E5] 


[E6] 



Individual Student Reports 

Individual student reports were given to [E7] 
Parents 

Schools and teachers 

Individual student results were expressed 
in terms of [E8] 

State’s achievement standards 

Scores 

Percentiles 

Interpretive guidance was developed for [E9] 
School-level administrators 
Teachers 
Parents 
Students 

Reports given to parents included the following 
information [El 0] 

Performance/achievement level 
Scores 

Standard/strand breakouts 
Indicator/benchmark breakouts 
Performance/achievement level descriptors 
Sample test items 



r \ 

State department of education 
website 

[website] 

V ) 



V Check marks indicate categories reported by the state from among those possible. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research, National 
Study on Alternate Assessments, state data summaries for school year 2006-07 based on document review and state verification. 

[STATE] Alternate Assessment for the 2006-07 School Year 
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[State] 



Participation and Proficiency (2006-07) 



Ways that students with lEPs participated in state accountability assessments 



Reading/Language Arts 




Percent 

□Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 1 

□ Exempt/absent 



Mathematics 




Percent 

□ Alternate assessment based on alternate 
achievement standards 

□ other assessments (regular assessments with 
or without accommodations) 1 

□ Exempt/absent 



Percentage of students who scored proficient or above on state accountability 
assessments 2 



Reading/Language Arts 




Percent 



Mathematics 




0 20 40 60 80 100 



Percent 



□ Students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ Students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



□ students with an IEP taking an alternate assessment 
based on alternate achievement standards 

□ Students with an IEP taking a regular assessment 
with or without accommodations 

□ All students including students with lEPs 



Percentage of students in the grades assessed who scored proficient or above on an 
alternate assessment based on alternate achievement standards 



Grades 


Reading/language arts 


Mathematics 


3-5 


0.60 


0.60 


6-8 


0.56 


0.43 


9-12 


0.54 


0.44 



1 For some states these included an alternate assessment based on grade-level achievement standards. 

2 Of all students who received valid scores on the specific types of accountability assessment. 

NOTE: Participation percentages may not sum to 100 because of rounding or data deficiencies. 

SOURCE: Thurlow, M.L., Altman, J.R., and Vang, M. (in press). Annual Performance Reports: 2006-2007 State Assessment 
Data. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes; U.S. Department of Education, 
Office of Planning, Evaluation and Policy Development, ED Facts - Consolidated State Performance Report for SY 2006-07, 
Washington, DC, 2008. 
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A. Overview 

A1 Alternate assessment 1 title 

This item asked for the name of the alternate assessment being used during the 
2006-07 school year. One state (Michigan) used two alternate assessments to assess students 
with significant cognitive disabilities. The names of the states’ alternate assessments are 
reported by state in the individual state profiles and in table A1 in the NSAA National Profile, 
appendix B. 

A2 Purposes of alternate assessment 

This item asked for the stated purposes and goals for the alternate assessment, in addition to 
meeting accountability requirements set by federal law. This was an open-ended item, and the 
following response categories emerged during coding. Multiple responses were possible. 

1 . Evaluate programs - This category was coded when the state specifically mentioned 
program evaluation as a purpose of the alternate assessment. 

2. Guide classroom instruction - This category was coded when the state reported that 
results of the assessment were intended to inform and refine classroom instruction for 
students with significant cognitive disabilities. 

3. Measure student progress/performance toward state standards - This category 
was coded when the state reported that measurement of individual student learning 
outcomes within the context of state accountability and state standards was a purpose 
of the alternate assessment. 

4. Assess student access to content standards/general curriculum - This category 
was coded when the state reported that evaluating access to the general education 
academic content standards for students with significant cognitive disabilities was a 
purpose of the alternate assessment. 

5. Assess individual students’ strengths/weaknesses - This category was coded when 
state reported that the gathering of information to inform instruction and measure the 
performance of individual students was a purpose of the alternate assessment. 

6. Document academic achievement - This category was coded when the state reported 
that documenting academic achievement and/or providing reports of student academic 
achievement to parents was a purpose of the alternate assessment. 

7. Measure student progress toward IEP goals - This category was coded when the state 
reported that a purpose of the alternate assessment was to inform IEP development or 
document whether IEP goals were or were not met. 

A3 Alternate assessment approaches (structures/types of items used) 

This item characterized the approaches states reported using for their 2006-07 alternate 
assessments. This was a multiple-choice item, and multiple responses were possible for states 
that used a combined approach (e.g., a series of performance tasks/events as well as submitted 
portfolios). 

1 . Rating scale/checklist 

2. Portfolio/body of evidence 

3. Performance task/events 

4. Multiple choice/constructed response 

1 Throughout the text, alternate assessment based on alternate academic achievement standards is referred to as 
“alternate assessment.” 
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A4 What content areas were included in the alternate assessment? 



This multiple-choice item asked for the specific content areas that were addressed by the state’s 
alternate assessments. Multiple responses were possible. 



1. Reading/language arts 

2. Mathematics 

3. Science 

4. Social studies 

5. Functional skills 



A5 Grades assessed 



This multiple-choice item asked for the specific grades (3 to 12) in which the state assessed 
students using the alternate assessment for measuring adequate yearly progress (AYP). 
Multiple responses were possible. 



1 . Grade 3 

2. Grade 4 

3. Grade 5 

4. Grade 6 

5. Grade 7 

6. Grade 8 

7. Grade 9 

8. Grade 10 

9. Grade 11 

10. Grade 12 



A6 What was the time frame within which the alternate assessment occurred? 

This multiple-choice item asked about the time frame of the administration of the alternate 
assessment by providing four mutually exclusive response options. 

1 . One day to 2 weeks 

2. More than 2 weeks to 1 month 

3. More than 1 month to 2 months 

4. More than 2 months 

A7 How many state content standards were there for reading/language arts? On how 
many content standards in reading/language arts were students with significant 
cognitive disabilities assessed using the alternate assessment? 

Two related items were investigated together: the number of general education content 
standards the state had in place for reading/language arts and on how many of those standards 
students with significant cognitive disabilities were assessed using the alternate assessment. 

States used different terms to refer to various levels of their system of general education content 
standards. For this item, the term “content standard” was used to refer to the highest level in 
a hierarchy of skills and knowledge, of which there were only a limited number (typically 1 0 
or fewer) for each content area. Although states often articulated additional subdomains of 
skills and knowledge, often down to deeper levels of specificity that described actual student 
performance, tasks, and/or activities, those levels are not reported here. 
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The second part of this item asked for the number of general education content standards 
on which students with significant cognitive disabilities were assessed by the state using an 
alternate assessment. In some states, each general education content standard was addressed 
in the alternate assessment in a way thought to be appropriate for students with significant 
cognitive disabilities. In other states, only a portion of the general education content standards 
were addressed in the alternate assessment. 

The results are expressed in the profiles as number of content standards on which students 
were assessed by the alternate assessment out of the total number of general education 
content standards in reading/language arts. 

A8 How many state content standards were there for mathematics? On how many 
content standards in mathematics were students with significant cognitive 
disabilities using the alternate assessment assessed? 

Two related items were investigated together: the number of general education content 
standards the state had in place for mathematics and on how many of those standards students 
with significant cognitive disabilities were assessed using the alternate assessment. 

States used different terms to refer to various levels of their system of general education content 
standards. For this item, the term “content standard” was used to refer to the highest level in 
a hierarchy of skills and knowledge, of which there were only a limited number (typically 10 
or fewer) for each content area. Although states often articulated additional subdomains of 
skills and knowledge, often down to deeper levels of specificity that described actual student 
performance, tasks, and/or activities, those levels are not reported here. 

The second part of this item asked for the number of general education content standards 
on which students with significant cognitive disabilities were assessed by the state using an 
alternate assessment. In some states, each general education content standard was addressed 
in the alternate assessment in a way thought to be appropriate for students with significant 
cognitive disabilities. In other states, only a portion of the general education content standards 
were addressed in the alternate assessment. 

The results are expressed in the profiles as number of content standards on which students 
were assessed by the alternate assessment out of the total number of general education 
content standards in mathematics. 

A9 Alternate assessment developer 

This item asked who was involved in the development of the alternate assessment. This was 
an open-ended item, and the following response categories emerged during coding. Multiple 
responses were possible. 

1 . Assessment company 

2. Research company/university/independent researcher 

3. Technical assistance provider (e.g., regional resource center) 

4. State personnel 

5. Parents 

6. Stakeholders 
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A10 Who administered/assembled the alternate assessment? 2 

This item asked who was involved in administering/assembling the alternate assessment. 

Multiple responses were possible. 

1 . The student’s special education teacher. 

2. A certified educator who was not the student’s teacher- This response category 
was coded when members of an assessment team, other classroom teachers, the 
student’s IEP team, or other support staff at the school or district level were allowed to 
administer or assemble the assessment but the student’s teacher was not involved in 
the assessment administration. 

3. Paraprofessionai - This response category was coded when aides or nonlicensed 
assistants were allowed to administer or assemble the alternate assessment. 

All Who scored the alternate assessment? 3 

This item asked who was allowed to score the alternate assessment. Multiple responses were 

possible. 

1 . Student’s classroom teacher. 

2. School- or district-based educator- This response category was coded when the 
scorer was not the student’s teacher but someone designated by the school or district 
administration, such as another teacher, IEP team member, counselor, or related- 
services personnel. 

3. State or state-contracted scorer- This response category was coded when the scorer 
was someone who did not work at the student’s school and served as a state agent 

in scoring the assessment, such as a test vendor staff member or an individual who 
served at a scoring “camp.” 

4. Machine scored - This response category was coded when student performance was 
evaluated electronically and not by the teacher or any other individual. This differed 
from instances in which a machine did the final tabulation of results or applied formulas 
to the results of individual scoring interpretations. 

5. Paraprofessionai - This response category was coded when aides or nonlicensed 
assistants were allowed to score the alternate assessment. 



2 Variables A10, D3, D4, D8, and E5 were derived from a series of open-ended items that asked about policies and 
procedures for administering and scoring the alternate assessment. These items were combined and organized 
into topic areas and then coded. 

3 Variables All, El, E2, and E6 were derived from a series of open-ended items that asked about the state’s 
guidelines, rules, and procedures for scoring the alternate assessment. These items were combined and 
responses were organized into multiple topic areas and specific response categories for each topic area during 
coding. The topic areas included information on who scored the assessment, how they were trained, and how 
conflicts in scoring were resolved if they arose. 
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B. Alternate Achievement Standards 

B1 Who was involved in creating the alternate achievement standards for students 

with significant cognitive disabilities for reading/language arts and mathematics? 

This multiple-choice item asked who was involved in creating the alternate achievement 
standards for students with significant cognitive disabilities in reading/language arts and 
mathematics. The original data collection included separate items for reading/language arts 
and mathematics in different grade spans (3-8 and 10-12), but there was sufficient overlap in 
responses to allow for reporting in aggregate. If a type of participant was involved in developing 
standards for any of the academic areas or grades, it was counted as a positive response for 
the state. Multiple responses were possible. 

1 . State special education staff 

2. State assessment staff 

3. State instruction and curriculum staff 

4. Test vendor 

5. Outside experts 

6. Special education teachers 

7. General education teachers 

8. Content specialists 

9. School psychologists/counselors 

10. School/district/state administrators 

11. Parents 

12. Other 

B2 Standard-setting methodologies used to develop alternate achievement standards 

This open-ended item asked about the process used to develop the alternate achievement 
standards (the descriptors and cut scores) in reading/language arts and mathematics across 
all the grades assessed. Responses were coded according to a list of possible approaches 
described in the literature (see table B-1). Multiple response codes were possible. 

Table B-1. Summary of standard-setting methodologies 



Methodology 


Summary description 


Test formats that work 
with this methodology 


1 . Modified 
Angoff 


Panelists estimate the percentage of minimally proficient students 
at each performance level who are expected to answer correctly/be 
able to do each test item; these individual estimates are summed to 
produce an overall percentage of items correct that correspond to 
the cut score for that level. 


Assessments with multiple 
items that are scored 
right/wrong. 


2. Extended 
Angoff 


Intended for open-ended items scored with a multiple-point rubric. 
Panelists determine the mean score that 100 minimally proficient 
students at each performance level would receive on this item. 
Summing the estimate across items produces the cut score. 


Assessments with open- 
ended items. 


3. Yes/No 
Method 


Rather than estimate a percentage, panelists simply determine 
whether or not a borderline student would be likely to answer 
correctly/be able to do each test item. Summing the number of 
“yeses” across items produces the cut score. 


Assessments that include 
items that are scored 
right/wrong or checklists. 
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Methodology 


Summary description 


Test formats that work 
with this methodology 


4. Bookmark or 
Item Mapping 


Panelists mark the spot in a specially constructed test booklet 
(arranged in order of item difficulty) where minimally proficient (or 
advanced) students would be able to answer correctly the items 
occurring before that spot with a certain probability. 


Assessments with multiple 
items that are scored 
right/wrong or with short 
rubrics. 


5. Performance 
Profile 
Method 


Panelists mark the spot in a specially constructed booklet of 
score profiles (arranged from lowest to highest total points) that 
designates sufficient performance to be classified as proficient. 

Each score profile uses a pictorial bar graph to display the student’s 
performance on each task of the assessment, and two to five 
profiles are shown for each raw score point. 


Assessments containing 
open-ended items, usually 
performance tasks, where 
it is difficult to provide 
samples of student work 
to show. 


6. Reasoned 
Judgment 


Panelists divide a score scale (e.g., 32 points) into a desired 
number of categories (e.g., four) in some way (equally, larger in the 
middle, etc.) based on expert judgment. 


Assessments that result in 
one overall score. 


7. Judgmental 
Policy 
Capturing 


Panelists determine which of the various components of an overall 
assessment are more important than others, so that components or 
types of evidence are weighted. 


Assessments that contain 
multiple components. 


8. Body of Work 


Panelists examine all of the data for a student and use this 
information to place the student in one of the overall performance 
levels. Standard setters are given a set of papers that demonstrate 
the complete range of possible scores from low to high. 


Assessments that consist 
primarily of performance 
tasks or one general body 
of evidence, such as a 
portfolio. 


9. Contrasting 
Groups 


Teachers separate students into groups based on their observations 
of the students in the classroom; the scores of the students are 
then calculated to determine where scores will be categorized in the 
future. 


Because this method 
is not tied to the test, it 
works with almost any test 
that results in an overall 
score. 


10. Item- 
Descriptor 
Matching 


Panelists determine what a student must know and be able to do to 
answer an item correctly. The panelists match these item-response 
requirements to a performance level descriptor. As panelists match 
items to the descriptors, sequences of items emerge in which some 
items match more closely, and cut scores are determined depending 
on patterns. 


Assessments that include 
dichotomously scored 
and polytomously scored 
items. 


11. Dominant 
Profile 
Method 


This method creates a set of decision rules to be used when tests 
are scored on several dimensions, such as performance, progress, 
generalization, and complexity, to determine rules for the cut 
score, describing whether there needs to be a minimum score on 
each dimension, on the total test, or some combination. It requires 
panelists to state exactly whether a high score on one dimension 
can compensate for a low score on another. The panelist’s task 
is to become familiar with the meaning of each dimension and to 
specify rules for determining which combinations of scores on these 
dimensions represent acceptable performance and which do not. 


Tests that are scored on 
several dimensions, such 
as performance, progress, 
generalization, and 
complexity. 



SOURCE: Cizek (2001); Ferrara, Perie, and Johnson (2002); Livingston and Zieky (1982); Perie (2007); Roeber (2002). 
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B3 What were the names for the advanced, proficient, and basic achievement levels 
for students being assessed based on alternate achievement standards for 
reading/language arts and mathematics? 

NCLB requires achievement standards to describe three levels of achievement — advanced, 
proficient, and basic. States often develop more than the required three levels and often apply 
different terminology to those levels. This open-ended item yielded specific information on the 
names of the various achievement levels for reading/language arts and for mathematics in each 
state. The names of the achievement levels that states adopted are reported by state in the 
individual state profiles and in table B3 in the NSAA National Profile, appendix B. 

B4 What descriptors applied to each achievement level for students being assessed 
based on alternate achievement standards for reading/language arts and 
mathematics? 

This open-ended item asked for the descriptor for each achievement level in reading/language 
arts and mathematics. The uniqueness of the descriptors does not allow for comparison across 
states. However, it was possible to code the states into mutually exclusive categories based on 
the degree to which descriptors were specific to grade levels or grade spans. 

1 . The same descriptors applied to all grades tested. 

2. The same descriptors applied to grade spans tested. 

3. Descriptors were unique for each grade tested. 

An example of a proficient-level descriptor is included in each state profile for illustrative 
purposes. 

B5 What cut scores were developed for reading/language arts and mathematics? 

This open-ended item asked what cut scores were developed for alternate achievement levels 
across grade levels in reading/language arts and mathematics. The following mutually exclusive 
response categories emerged during coding. 

1 . Unique cut scores for each grade 

2. Unique cut scores for grade spans 

3. One set of cut scores for all students 

4. Other approaches (e.g., applying a rubric to determine proficiency level) 
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C. Technical Quality 4 

Cl, C2, C3, C4 Who was involved in reviewing the technical characteristics of validity, 
reliability, alignment, and fairness of the alternate assessment? 

Four multiple-choice items asked about who was involved in reviewing the validity, reliability, 
alignment, and fairness of the alternate assessment. Multiple responses were possible. 

1 . State special education staff 

2. State assessment staff 

3. State instruction and curriculum staff 

4. Test vendor 

5. Outside experts 

6. Special education teachers 

7. General education teachers 

8. Content specialists 

9. School psychologists/counselors 

10. School/district/state administrators 

11. Parents 

12. Other 

C5 Did the state document the validity of the alternate assessment in terms of scoring 
and reporting structures consistent with the subdomain structures of its content 
standards? 

This open-ended item asked whether the state had documented that the scoring and reporting 
structures reflected the knowledge and skills that students were expected to master and 
identified the nature of the evidence provided. If the reading standards were divided into certain 
subdomains/areas/categories, then evidence of the scoring and reporting structures should be 
divided into the same subdomains/areas/categories. The following categories emerged during 
coding, and responses to this item were mutually exclusive. 

1 . Yes, with evidence provided to the research team - This response category was 
coded when the state provided evidence that the depth and breadth of the standards 
were reflected or built into the scoring and reporting structures. Documents provided 
evidence that student performance was reported at the subdomain level, not just by 
content area. In other words, the state produced the scores for subdomain categories 
(i.e. , standards/benchmarks/indicators), which were the same subdomain categories as 
those in the content standards. In cases where states provided evidence to the research 
team, it was in the form of scoring and reporting documents. An alignment study on its 
own would not be sufficient evidence to code this response category; rather, there must 
be evidence that the scoring and reporting were consistent with the subdomains of the 
content standards. 

2. Yes, but evidence was not provided to the research team - This response category was 
coded when the state claimed validity based on scoring and reporting structures, but the 
evidence was part of an internal, nonpublic report and was not available for examination 
by the research team. 

4 For the technical quality variables reported here, when evidence was provided to the research team, the evidence 
was examined to describe and classify it. It was beyond the scope of this study to summarize the findings of the 
evidence or to evaluate its quality or rigor. 
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3. No - The state did not claim or document the validity of the alternate assessment in 
terms of scoring and reporting structures consistent with the subdomain structures of its 
content standards. 

C6 Did the state document the validity of the alternate assessment in terms of test 
and item scores related to internal or external variables as intended? 

This open-ended item asked whether the state had documented the validity of test and item 
scores based on analysis of the relationship of test and item scores to one another (internal 
validity) or to other measures (external validity) and the nature of the evidence provided. The 
following response categories emerged during coding and were mutually exclusive. 

1 . Yes, formal study conducted - This category was coded when the state reported that 
a formal study or expert panel review was conducted, and evidence of the study was 
provided to the research team. The evidence may have been part of an internal or 
external study and was reported publicly or provided to the research team. 

2. Yes, but evidence was not provided to the research team - This response category was 
coded when the state reported that an internal study had been conducted or a formal 
study was in progress. The evidence may have been part of a plan or a study that was 
under way, and/or the evidence was part of an internal, nonpublic report. These reports 
were not available for examination by the research team. 

3. Yes, but no formal study was conducted - This response category was coded when the 
state reported in an explanation or through anecdotes that validation occurred through 
a committee process or an internal review, but no formal study was conducted. In these 
cases, the type of evidence was reported in the profile as a “committee process or 
internal review.” 

4. No - The state did not document the validity of the alternate assessment in terms of test 
and item scores related to internal or external variables as intended. 

C7 What evidence supported the validity argument in terms of test and item scores 
related to internal or external variables as intended? 

This open-ended item asked about the types of formal analyses reported when the state had 
conducted a formal validity study of the test and item scores related to internal or external 
variables (see C6, response category “yes, formal study conducted”). Evidence may have 
included an indication that there were other assessments reported, such as standardized 
measures appropriate for students with significant cognitive disabilities that confirmed the 
results for similar students (external validity). Alternatively, the state may have provided 
statistical evidence that indicated the test items correlated with a total score in the same 
way (internal validity). The following types of evidence emerged during coding, and multiple 
responses were possible. 

1 . Correlational study indicating validity 

a. Internal item-to-item analysis 

b. Correlational analysis using external measures (e.g., teacher grades, Academic 
Competence Evaluation Scales [ACES], different test) 

2. Other type of analysis reported or specific analytic strategy/approach not detailed 



Appendix B: Explanation of Item Codes 



B-19 




NSAA State Profiles 



C8 Did the state document the validity of the alternate assessment in terms of 

purposes of the assessment, delineating the types of uses and decisions most 
appropriate and the assessment results consistent with the purposes? 

This open-ended item asked whether the state had documented the consistency of purposes 
of the assessment with the decisions made based on assessment results and the nature of 
the evidence provided. The following response categories emerged during coding and were 
mutually exclusive. 

1 . Yes, formal study conducted - This response category was coded when the state 
reported that a formal study or expert panel review was conducted. The evidence may 
have been part of either an internal or an external study, and the results were reported 
publicly or were provided to the research team. 

2. Yes, but evidence was not provided to the research team - This response category was 
coded when the state reported that an internal study had been conducted or a formal 
study was in progress, and/or the evidence was part of an internal, nonpublic report. 
These reports were not available for examination by the research team. 

3. Yes, but no format study was conducted - This response category was coded when the 
state reported that a validation was planned or under way and offered explanation or 
anecdotes that this type of validation had been done as part of a committee process, 
but no formal study was conducted. In these cases, the type of evidence was reported 
in the state profile as “anecdotal or committee process.” 

4. No - The state did not document the validity of the alternate assessment in terms of the 
purposes of the assessment. 

C9 What evidence supported the validity argument in terms of purposes of the 

assessment, delineating the types of uses and decisions most appropriate and the 
assessment results consistent with the purposes? 

This open-ended item asked about the types of formal analyses reported by the state when it 
had conducted a formal validity study on the consistency of the purposes and uses of the results 
of the assessment (see C8, response category “yes, formal study conducted”). The following 
response categories emerged during coding, and multiple responses were possible. 

1 . Survey about the relationship between the purposes of the assessments and decisions 
made 

2. Alignment study 

3. Field tests/pilot tests 

4. Construct validity analysis 

5. Analytic review of outcomes 

6. State monitoring/program review 
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CIO Did the state document the validity of the alternate assessment in terms of the 
assessment system’s producing intended and/or unintended consequences? 

This open-ended item asked whether the state documented the intended and/or unintended 
consequences of the assessment and the degree to which the determination of validity had 
been documented. The following response categories emerged during coding and were 
mutually exclusive. 

1 . Yes, formal study conducted - This response category was coded when the state 
reported that an internal or external study or expert panel review had been conducted, 
and the report was available publicly or provided to the research team. 

2. Yes, but evidence was not provided to the research team - This response category 
was coded when the state reported that a plan or a study was under way, or the 
evidence was part of an internal, nonpublic report. These reports were not available for 
examination by the research team. 

3. Yes, but no formal study was conducted - This response category was coded when the 
state provided an explanation or anecdotes regarding a committee review process, but 
no formal study was conducted. In these cases, the type of evidence was reported in 
the state profile as “anecdotal or committee process.” 

4. No - This response category was coded when the state had not documented the 
validity of the alternate assessment in terms of the assessment system’s producing 
intended and/or unintended consequences. 

C11 What evidence supported the validity argument in terms of the assessment 
system’s producing intended and/or unintended consequences? 

This open-ended item described the types of evidence provided to document validity in terms 
of intended and/or unintended consequences (see CIO, response category “yes, formal study 
conducted”). Evidence could include arguments or empirical evidence that demonstrated the 
direct or indirect consequences of taking the alternate assessment for students with significant 
cognitive disabilities, including those that were intended or unintended, positive or negative. 
Some items that were commonly addressed with this type of validity study were: Did the 
student learn more or less as a result of taking the assessment? Was an appropriate amount 
of preparation spent on the assessment? Did the assessment affect the student emotionally 
or functionally in some way? Did the assessment affect teacher understanding of the student’s 
educational needs? Did the assessment change how teachers teach? The following response 
categories emerged during coding. Multiple responses were possible. 

1 . Survey- This response category was coded when the state provided studies using 
surveys of teachers, parents, or other school staff as evidence. 

2. Public reports - This response category was coded when the state provided published 
reports or other accounts of the consequences of the assessment. 

3. Other post hoc data collection/analysis - This response category was coded when the 
state provided evidence of other types of data collection and analysis. 
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C12 Did the state document the validity of the alternate assessment in terms of 
measurement of construct relevance? 

This open-ended item asked whether the state had documented the construct relevance of 
the assessment (i.e., whether it measured the behavior or knowledge of interest, whether it 
measured only the standards and content appropriate to the age or grade of the assessed 
student and not information extraneous to the construct). Additionally, the item asked about the 
degree to which the determination of validity had been documented. The following response 
categories emerged during coding and were mutually exclusive. 

1 . Yes, formal study conducted - This response category was coded when the state 
reported that an internal or external study or expert panel review had been conducted, 
and the report was available publicly or provided to the research team. 

2. Yes, but evidence was not provided to the research team - This response category 
was coded when the state reported that a plan or a study was under way, or the 
evidence was part of an internal, nonpublic report. These reports were not available for 
examination by the research team. 

3. Yes, but no formal study was conducted - This response category was coded when the 
state provided an explanation or anecdotes regarding a committee review process, but 
no formal study was conducted. In these cases, the type of evidence was reported in 
the state profile as “anecdotal or committee process.” 

4. No - This response category was coded when the state had not documented the 
validity of the alternate assessment in terms of construct relevance. 

C13 What evidence supported the validity argument in terms of measurement of 
construct relevance? 

This open-ended item described the types of evidence provided to document the measurement 
of construct relevance (see C12, response category “yes, formal study conducted”). Evidence 
could include arguments or empirical evidence that demonstrated that the behavior or 
knowledge of interest was measured as intended. The following response categories emerged 
during coding, and multiple responses were possible. 

1 . Statistical analyses - This response category was coded when the state provided 
evidence of conducting factor analysis, item-to-item analysis, and/or correlational 
studies across tests of similar constructs. 

2. Construct analyses - This response category was coded when the state provided 
evidence of alignment studies or other reviews by trained judges regarding the construct 
of the assessment. 

C14 Did the state document the validity of the alternate assessment in terms of grade- 
level equating? 

This open-ended item asked whether the state had documented the validity of the alternate 
assessment in terms of grade-level equating (i.e., the extent to which assessment items and 
tasks were calibrated within and across grade levels). Additionally, the item asked about the 
degree to which the determination of validity had been documented. The following response 
categories emerged during coding and were mutually exclusive. 

1 . Yes, formal study conducted - This response category was coded when the state 
reported that an internal or external study or expert panel review had been conducted, 
and the report was available publicly or provided to the research team. In these cases, 
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the type of evidence was reported in the profile as a “formal study or expert panel 
review.” 

2. Yes, but evidence was not provided to the research team - This response category 
was coded when the state reported that a plan or a study was under way, or the 
evidence was part of an internal, nonpublic report. These reports were not available for 
examination by the research team. 

3. Yes, but no formal study was conducted - This response category was coded when the 
state provided an explanation or anecdotes regarding a committee review process, but 
no formal study was conducted. In these cases, the type of evidence was reported in 
the state profile as “anecdotal or committee process.” 

4. No - This response category was coded when the state had not documented the 
validity of the alternate assessment in terms of grade-level equating. 

5. Not appropriate for this type of assessment - This response category was coded 
when the state reported that grade-level equating was not appropriate for the type of 
assessment used and the assessment approach did not meet the assumptions needed 
to conduct this type of analysis. 

Cl 5 Had the state content standards been extended or adapted to provide access for 
students with significant cognitive disabilities? 

This item asked, at a summary level, whether the state had developed an augmented or 
supplementary list of “extended” standards that presented at the appropriate level of challenge 
for students with significant cognitive disabilities to clarify the relationship between the academic 
content standards and the alternate assessment, and that allowed such students access to 
state curricular content. State responses were coded using the following options: 

1 . Yes 

2. No 

Cl 6 How did the extended content standards map to the state content standards? 

This item asked about the extent of the linkage between the state’s “extended” content 
standards and the state’s general education grade-level content standards. This was an open- 
ended item, and the following response categories emerged during coding. Multiple response 
codes were possible. 

1 . General link to state content standards - This response category was coded when 
the state reported that the extended content standards were linked at a general level 
with the state’s content standards. This refers to the broad concept or first level in the 
hierarchy of the state’s standards. For example, one state required that the content 
domains of language arts, mathematics, and science be addressed but did not link to 
content area strands or grade-level competencies within those content areas. 

2. Grade or grade span - This response category was coded when the state reported that 
the state’s standards had been expanded, defined, or redefined in terms of grade levels 
or grade spans to create the extended standards. For example, one state specified, 
within each content domain, the content strands that should be addressed in 

grades 3-8 and then in high school. 

3. Expanded benchmarks - This response category was coded when the state reported 
that its expanded standards provided greater specificity regarding the expectations 
for students with significant cognitive disabilities. These were downward extensions of 
the standards, which may have been referred to as expanded benchmarks, extended 
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standards, essences, or dimensions. Expanded benchmarks might include information 
about the levels of complexity or depth of knowledge and describe the “essence” of 
standards or an “extension” to access points. For example, in one state the general 
‘indicator’ “Demonstrate the ability to use a variety of strategies to derive meaning from 
texts and to read fluently” was expanded to “Understand how print is organized.” 

4. Alternate indicators or tasks - This response category was coded when the state 
reported that it had developed levels of specification that described activities, tasks, 
or how student performances might be structured, often referred to as performance 
indicators, indicator tasks, indicator activities, or alternate performance indicators 
(APIs). For example, in one state the learning standard “Identify and represent common 
fractions (1/2, 1/3, 1/4) as parts of wholes, parts of groups, and numbers on the number 
line was linked to the following activities at different entry points: (1) Understand whole 
and half; (2) Manipulate objects to make two objects from one; (3) Manipulate whole 
objects to make two, three, or four parts of a whole; (4) Manipulate up to four parts of 
an object to assemble a whole; and (5) Identify and compare parts of a whole (quarters, 
thirds, halves) and determine relative size of each (1/2, 1/3, 1/4) using manipulatives.” 

Cl 7 Did the state document the reliability of the alternate assessment in terms of 
variability across groups? 

This open-ended item asked whether the state had documented the reliability of the alternate 
assessment in terms of differences in the performances of students in the various NCLB-defined 
groups (e.g., race/ethnicity, economically disadvantaged, limited English proficient). Additionally, 
the item asked about the degree to which the determination of reliability had been documented 
and reported. The following response categories emerged during coding and were mutually 
exclusive. 

1 . Yes, formal study conducted - This response category was coded when the state 
reported that an internal or external study or expert panel review had been conducted, 
and the report was available publicly or provided to the research team. 

2. Yes, but evidence was not provided to the research team - This response category 
was coded when the state reported that a plan or a study was under way, or the 
evidence was part of an internal, nonpublic report. These reports were not available for 
examination by the research team. 

3. Yes, but no formal study was conducted - This response category was coded when the 
state provided an explanation or anecdotes regarding a committee review process, but 
no formal study was conducted. In these cases, the type of evidence was reported in 
the state profile as “anecdotal or committee process.” 

4. No - This response category was coded when the state had not documented the 
reliability of the alternate assessment in terms of variability across groups. 

5. Not appropriate for this type of assessment - This response category was coded when 
the state reported that analyzing the reliability in terms of the variability across groups 
was not appropriate for this type of assessment and the assessment approach did not 
meet the assumptions needed to conduct this type of analysis. 
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C18 What evidence supported the reliability argument in terms of variability across 
groups? 

This open-ended item described the types of evidence provided to support the reliability of 
the alternate assessment in terms of variability across groups (see Cl 7, response category 
“yes, formal study conducted). The following response categories emerged during coding, and 
multiple responses were possible. 

1 . NCLB group statistical analyses conducted - This response category was coded when 
the state provided evidence of differential item functioning (DIF) analyses, consistency 
reliability, and/or test-retest reliability. 

2. Review of disability group results - This response category was coded when the state 
provided evidence of a published review by an expert panel or review group. 

C19 Did the state document the reliability of the alternate assessment in terms of 
internal consistency of item responses? 

This open-ended item asked whether the state had documented that there was consistency 
between scores on particular groups of items and the total test score and that scores on one 
item were consistent with scores on other items that were measuring the same construct. These 
reliability test results should provide statistical evidence of item consistency; if the state reported 
having conducted a study, it should be of a statistical nature and statistical results should be 
evident. Additionally, the item asked about the degree to which the determination of reliability 
had been documented and reported. The following response categories emerged during coding 
and were mutually exclusive. 

1 . Yes, formal study conducted - This response category was coded when the state 
reported that an internal or external study or expert panel review had been conducted, 
and the report was available publicly or provided to the research team. In these cases, 
the type of evidence was reported in the state profile as “formal study or expert panel 
review.” 

2. Yes, but evidence was not provided to the research team - This response category 
was coded when the state reported that a plan or a study was under way, or the 
evidence was part of an internal, nonpublic report. These reports were not available for 
examination by the research team. 

3. Yes, but no formal study was conducted - This response category was coded when the 
state provided an explanation or anecdotes regarding a committee review process, but 
no formal study was conducted. In these cases, the type of evidence was reported in 
the state profile as “anecdotal or committee process.” 

4. No - This response category was coded when the state had not documented the 
reliability of the alternate assessment in terms of internal consistency of item responses. 

5. Not appropriate for this type of assessment - This response category was coded when 
the state reported that analyzing the reliability in terms of internal consistency of item 
responses would not be appropriate for this type of assessment and the assessment 
approach did not meet the assumptions needed to conduct this type of analysis. 
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C20 Did the state document the reliability of the alternate assessment in terms of 
interrater consistency in scoring? 

This open-ended item asked whether the state had conducted statistical procedures to 
examine the consistency and reliability of scoring between and among scorers. Additionally, 
the item asked about the degree to which the determination of reliability had been documented 
and reported. The following response categories emerged during coding and were mutually 
exclusive. 

1 . Yes, formal study conducted - This response category was coded when the state 
reported that an internal or external study or expert panel review had been conducted, 
and the report was available publicly or provided to the research team. 

2. Yes, but evidence was not provided to the research team - This response category 
was coded when the state reported that a plan or a study was under way, or the 
evidence was part of an internal, nonpublic report. These reports were not available for 
examination by the research team. 

3. Yes, but no formal study was conducted - This response category was coded when the 
state provided an explanation or anecdotes regarding a committee review process, but 
no formal study was conducted. In these cases, the type of evidence was reported in 
the state profile as “training documents or anecdotal.” 

4. No - This response category was coded when the state had not documented the 
reliability of the alternate assessment in terms of interrater consistency of scoring. 

C21 What evidence supported the reliability argument in terms of interrater 
consistency in scoring? 

This open-ended item described the types of evidence provided to support the reliability of 
the alternate assessment in terms of interrater consistency in scoring and to document that 
statistical procedures were used to examine the consistency of scoring between and among 
scorers (see C20, response category “yes, formal study conducted”). Evidence should 
demonstrate that the state analyzed the frequency with which scorers scored tests similarly, 
using interrater reliability analyses. The following response categories emerged during coding, 
and multiple responses were possible. 

1 . Statistical analysis conducted as part of training - This response category was coded 
when states reported calculating correlation coefficients, agreement percentages, 

or other analysis of scoring done as part of scorer training. States may also have 
established interrater consistency cut points that scorers must meet to obtain scorer 
certification during training. 

2. Statistical analysis conducted as part of actual scoring - This response category was 
coded when states reported calculating correlation coefficients, agreement percentages, 
or other analysis of assessment or field test scores. 
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C22 Had conditional standard errors of measurement (CSEMs) been reported for the 
alternate assessment? 

This set of items asked whether the state had analyzed the standard errors of measurement 
(SEMs) or conditional standard errors of measurement (CSEMs). The following response 
categories emerged during coding and were mutually exclusive. 

1 . Yes - This response category was coded when SEM/CSEM calculation procedures 
were reported by the state. 

2. No - This response category was coded when no evidence was provided that SEMs/ 
CSEMs were calculated or reported. 

3. Not appropriate for this type of assessment - This response category was coded when 
the state reported that the item was not appropriate for this type of assessment and 
the assessment approach did not meet the assumptions needed to conduct this type of 
analysis. 

C23 What was the initial process of aligning alternate achievement standards with the 
state content standards, and how was it validated? 

This open-ended item asked about the processes and methodologies the state used to align its 
alternate achievement standards with state content standards, as well as how this alignment 
was validated. The following mutually exclusive response categories emerged during coding. 

1 . A formal alignment study was conducted - This response category was coded when the 
state reported that an internal or external alignment study had been conducted, and it 
was reported publicly. Evidence may include one or more formal expert panel reviews 
or studies using methodologies such as Webb or LINKS. In these cases, the type of 
evidence was reported in the state profile as “formal study.” 

2. Alignment was reported, but no formal study was conducted - This response category 
was coded when states provided an explanation or anecdotes about a committee 
process to establish alignment, but no formal study was conducted. In these cases, the 
type of evidence was reported in the state profile as “anecdotal or committee process.” 

3. No alignment study was conducted - This response category was coded when the 
alternate achievement standards were not validated by an alignment study. 

C24 What ongoing procedures were used to maintain and improve alignment between 
the alternate assessment based on alternate achievement standards and state 
content standards over time? 

This item asked for the types of procedures the state used to monitor the alignment of the 
alternate assessment with state content standards and ensure that future alignment studies 
would be conducted. This was a multiple-choice item, and multiple responses were possible. 

1 . Internal alignment studies - This response category was coded when alignment studies 
were conducted by state assessment staff. 

2. External alignment studies - This response category was coded when outside experts 
conducted alignment studies. 

3. Other alignment studies - This response category was coded when an internal review 
was held or the details of the type of alignment study were not specified by the state. 

4. No alignment studies conducted - This response category was coded when the 
alternate achievement standards were not validated on an ongoing basis. 
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C25 Was there a process to ensure fairness in the development of the alternate 
assessment? 

This open-ended item asked whether the state used a formal process (a statistical validation 
process, a committee review, etc.) to ensure that students’ performance on the alternate 
assessment was not biased or influenced, for example, by native language, prior experience, 
gender, ethnicity, or disability. The following response categories emerged during coding and 
were mutually exclusive. 

1 . Yes, bias review conducted systematically and regularly - This response category 
was coded when the state reported that either qualitative or statistical analyses were 
conducted (bias review) to ensure fairness in the assessment. Assessments may have 
been reviewed by technical committees and/or expert panels, and the results were 
reported either internally or externally on a regular basis. 

2. Yes, bias review not conducted regularly - This response category was coded when 
the state reported that bias review was conducted formally or informally, typically in 
conjunction with assessment development or revision, but not on a regular basis. 
Statistical evidence was sporadic and not necessarily available publicly. 

3. No evidence of bias review- This response category was coded when the state did not 
provide evidence of a process to ensure fairness in the development of the alternate 
assessment. 

C26 What evidence supported the process to ensure fairness in the development of 
the alternate assessment? 

This open-ended item asked what types of evidence supported the process to ensure fairness 
in the development of the alternate assessment (see C25, response category “Yes, bias review 
was conducted systematically and regularly”). The following response categories emerged 
during coding, and multiple responses were possible. 

1 . Regularly scheduled bias review by experts 

2. Statistical analyses (e.g., differential item functioning [DIF] analysis) 

C27 Did the state document the validity of the alternate assessment in terms of 
implementation processes? 

This open-ended item asked whether the state had documented the validity of the alternate 
assessment in terms of implementation processes. Implementation processes included how 
the state informed districts and schools about the assessment and assessment procedures 
and how test administrators were trained. Validation of these processes might have occurred 
through a variety of means, including training, guidelines, manuals, monitoring, and follow-up 
analyses. The following response categories emerged during coding, and responses were 
mutually exclusive. 

1 . Yes, with evidence provided to the research team - This response category was coded 
when the state reported that formal studies or expert panel reviews were conducted on 
implementation processes, and evidence was part of documentation reported publicly or 
was provided to the research team. 

2. Yes, but evidence was not provided to the research team - This response category was 
coded when the state reported that this type of validation was planned or under way, 

or the evidence was part of an internal, nonpublic report of implementation processes. 
These reports were not available for examination by the research team. 
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3. Yes, but no formal study was conducted - This response category was coded when the 
state reported in an explanation or through anecdotes that validation of implementation 
processes occurred as part of a committee process, but no formal study was conducted. 
In these cases, the type of evidence was reported in the profile as “anecdotal or 
committee process.” 

4. No - The state did not claim or document the validity of the alternate assessment in 
terms of the implementation processes. 

C28 What evidence supported the validity argument in terms of implementation 
processes? 

This open-ended item asked about the types of evidence the state provided to support the 
validity of the alternate assessment in terms of implementation processes (see C27, response 
category “yes, with evidence provided”). The following response categories emerged during 
coding, and multiple responses were possible. 

1 . Training - This response category was coded when the state reported that it had 
developed teaching tools that included in-person, video, or online training for 
administration, scoring, and reliability. 

2. Administration manual/guide - This response category was coded when the state 
reported that it had developed manuals that provided directions, sample entries, 
protocols, and scoring rubrics. These manuals may have been available in hard copies 
or on websites. 

3. Monitoring - This response category was coded when the state reported that monitoring 
was conducted by the state agency, outside experts, citizen groups, or school-level 
administrators. These processes may have included sign-in verification by principals, 
test coordinators, or teachers. 

4. Post hoc data collection/analysis - This response category was coded when the state 
reported that reliability rescoring and examining of assessment results to determine 
fidelity were conducted. 



Appendix B: Explanation of Item Codes 



B-29 




NSAA State Profiles 



D. Eligibility and Administration 

D1 What were the guidelines for IEP teams to apply in determining when a child’s 
significant cognitive disability justified alternate assessment? 

This item asked about the eligibility criteria the state established to determine when the alternate 
assessment was appropriate for a student. This was an open-ended item, and the following 
response categories emerged during coding. Multiple responses were possible. 

1 . Had a severe cognitive disability (e.g., significant impairment of cognitive abilities, 
operates at a lower cognitive level). 

2. Required modified instruction (e.g., student required differentiated, intensive, and 
individualized instruction). 

3. Required extensive support for skill generalization (e.g., needed support to transfer 
skills to other settings, support to generalize learning to home/work/school/multiple 
settings). 

4. Required modified curriculum (e.g., student was unable to access general curriculum, 
general curriculum must be modified or presented at a lower cognitive level). 

5. Not based on disability category (decisions should not be based solely on disability 
category or other similar qualities). 

D2 What procedures were in place for informing parents when their child would be 
assessed using an alternate assessment? 

This item asked about the steps taken to inform parents (or guardians) that the student would 
be assessed using an alternate assessment, including the consequences of participation in 
this type of assessment — for example, the implications for graduation and the type of diploma 
a student would earn. This was an open-ended item, and the following response categories 
emerged during coding. Multiple responses were possible. 

1 . Parent signature was required - This response category was coded when the state 
reported that the signature of a parent or guardian was required specifically for 
permission for an alternate assessment to be conducted or that a signature on an IEP 
(that contained reference to an alternate assessment to be conducted) was required. 

2. Parents were provided written materials about the alternate assessment - This 
response category was coded when the state reported that parents received written 
materials about the alternate assessment. 

3. Nonspecific information provided. 

D3 How was assessment content selected? 

This item asked about the amount of input the state had in determining assessment content. 

The following mutually exclusive response categories emerged during coding. 

1 . All components - State determined academic content areas and strands, standards, 
benchmarks, and performance indicators/tasks - This response category was coded 
when the state determined the content areas, standards, benchmarks, or indicators 
assessed and no local input was allowed. 

2. Most components - State determined academic content areas, strands and standards, 
and the IEP team determined performance indicators/tasks - This response category 
was coded when the IEP team or teacher decided which tasks or academic indicators 
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comprised a student’s assessment or when teachers could choose from a task bank or 
develop their own. 

3. Some components - State determined only the academic content areas - This 

response category was coded when the IEP team or teacher decided which strands, 
standards, benchmarks, and tasks or indicators were assessed within academic content 
areas determined by the state. 

D4 How was the administration process monitored and verified? 

This item asked about how the administration process for the alternate assessment was 
monitored and verified and who was primarily responsible for the verification process. The 
following response choices emerged during coding, and multiple responses were possible 

1 . An observer/monitor was present - This response category was coded when a monitor 
was present for all administrations. 

2. A local or school-level reviewer confirmed proper administration of the 
assessment - This response category was coded when the assessment was reviewed 
by school-level staff who did not administer the actual assessment or assemble the 
portfolio. No monitor was present for the administration, but someone local confirmed 
that the assessment was administered properly. 

3. No independent verification process - This response category was coded when the test 
administrator was the sole person verifying assessment administration. 

D5 What procedures were followed in gathering performance evidence? 

This item asked about the flexibility that existed in the gathering of performance evidence — 
whether the state specified the required types of performance evidence, such as standardized 
tasks/test items/rating scales, the state provided guidance and instructions, or the teacher/lEP 
team made these decisions. The following response choices emerged during coding, and 
multiple responses were possible. 

1 . State required standardized tasks/test items/rating scales - This response category 
was coded when the state required evidence in the form of student responses on 
standardized tasks or test items or teachers were required to provide ratings of student 
performance. Work samples were not collected or submitted as evidence for scoring, 
and the scoring was based on performance tasks or teacher ratings of student skills. 

2. State provided instructions - This response category was coded when the state 
provided instructions on the types and amounts of evidence or data required from each 
student (certain types or formats of performance, such as video, documented student 
work, data sheets, or captioned photographs). 

3. Teacher/lEP team decided - This response category was coded when the teacher 
or IEP team determined the nature of evidence required for scoring, without state 
guidance, including instances where a local educator decided what could be scored for 
each indicator. This response category also was coded when the state used checklists 
with lEP-aligned tasks. 

D6 Describe the role of student work (videos, photographs, worksheets/products) in 
the alternate assessment. 

This item asked about the extent to which the state alternate assessment involved collecting 
samples of student work (or other evidence of class work performed by students). Specifically, it 
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examined what evidence of student work was considered for scoring the alternate assessment. 
This was an open-ended item, and the following response categories emerged during coding. 
Responses to this item were mutually exclusive. 

1 . Student work samples only - This category was coded when the state reported that 
portfolios and other collections of student work or bodies of evidence were included 
in the alternate assessment and submitted for scoring. The alternate assessment 
consisted entirely of collections of work samples or evidence of work produced by 
students (e.g., videos, photographs, worksheets, work products). 

2. Combination of work samples and other evidence - This category was coded when 
the state reported that the alternate assessment included a combination of student 
work samples and other assessment evidence, such as scores on on-demand tasks, 
checklists, or rating scales. 

3. No student work samples - This category was coded when the state reported that the 
alternate assessment used checklists and rating scales or other scoring mechanisms, 
such as scores on performance tasks, but did not require that evidence of student work 
be submitted to the state along with scores. 

D7 Did the assessment of student work (tasks or products) take place as part of the 
day-to-day instructional activities or “on demand”? 

This item asked whether the alternate assessment was embedded in daily classroom 
instruction or was an “on-demand” assessment. An on-demand assessment was one that was 
administered at an explicitly defined place and time and was separate from instruction, meaning 
that student performance and products were not derived from the teacher’s instructional plan 
and the classroom routine. On-demand assessments were typically standardized, given in the 
same format to all test takers, and scheduled in advance such that they supplanted instructional 
time. The mutually exclusive response categories that follow emerged during coding. 

1 . Part of day-to-day student instruction - This category was coded when the state 
reported that the alternate assessment involved a variety of activities that took place 
as part of daily instructional activities, including checklists that assess student work 
in progress, assessments that gather student work for portfolios, performance tasks 
embedded in instruction, and other assessments that are designed specifically to be 
part of the student’s daily instructional or learning routine. 

2. Separately from student’s daily work (on demand) - This category was coded when the 
state reported that the assessment supplanted instructional time and included multiple- 
choice assessments and standardized performance tasks/events. 

3. A combination of day-to-day and on-demand approaches - This category was coded 
when the state reported that the alternate assessment combined both approaches: 
activities during instructional time and on-demand activities. 

4. Based on teacher recollection of student performance - This category was coded 
when the state reported that the alternate assessment was a checklist or rubric and 
was completed without requiring supportive evidence of student work that teachers 
completed based on their expectations or recollections of student performance. 

D8 Describe the role of teacher judgment in the alternate assessment. 

This item asked about how the student’s classroom teacher and/or IEP team members were 
involved in determining (1) what content was assessed, (2) what student work products would 
be scored, (3) when and how the student would be assessed, and (4) who evaluated the 
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student’s performance or scored the state’s alternate assessment. This was an open-ended 
item, and the following response categories emerged during coding. Multiple responses were 
possible. 

1 . Teacher decided content to be assessed - This response category was coded 
when the state reported that the teacher determined some or all of the assessment 
content including selecting the standards to be assessed or the indicators to be used, 
determining the level of complexity of assessed tasks, and/or defining specific tasks. 

2. Teacher selected materials - This response category was coded when the state 
reported that the teacher was responsible for portfolio construction or assembly; 
selection of specific evidence, captioned pictures, anecdotal records, or videotape; or 
writing a student learning profile for the scorers. 

3. Teacher made decisions about administering the assessment - This response category 
was coded when the state reported that the teacher made decisions about some 

(but not necessarily all) of the following factors in the administration of the alternate 
assessment: timing and duration of the assessment, level of support or scaffolding to be 
provided, and/or when to administer the test within a testing window. 

4. Teacher interpreted/recorded student responses or scores - This response category 
was coded when the state reported that the teacher used a scoring rubric or checklist 
to determine the student scores and recorded student responses or scores on a report, 
including when the teacher was the final determiner of the student’s assessment score. 
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E. Scoring and Reporting 

El How many scorers scored the alternate assessment? 

This item asked about the number of scorers used to determine an individual’s score on the 
alternate assessment. Response categories were mutually exclusive. 

1 . One scorer 

2. Two scorers 

3. Three or more scorers 

E2 How were scoring conflicts resolved? 

This item asked about how the state resolved conflicts in scoring if they arose. The following 
mutually exclusive response categories emerged during coding. 

1 . A third person adjudicated - This response category was coded when a third person 
helped scorers come to agreement or ruled in favor of one or the other in disputes 
between two scorers. 

2. A third rater scored the alternate assessment - This response category was coded 
when a third score replaced the original scores or was combined with the first two 
scores for a new score. 

3. One person scored or scores were combined - This response category was coded 
when the state used only one scorer or different scores were simply averaged or 
combined. 

E3 What elements of student performance were used in scoring? 

This multiple-choice item asked about the state’s scoring criteria at the student level. Multiple 
responses were possible. 

1 . Accuracy of student response - This response category was coded when the 
correctness of a response or the production of student work that reflected the intended 
response of the assessment item or activity was a component of scoring. 

2. Ability to generalize across settings - This response category was coded when the 
student’s ability to perform a task in multiple settings or under differing conditions was a 
component of scoring. 

3. Amount of independence - This response category was coded when the degree 
of independence of the student’s response (or lack of prompting or scaffolding of a 
response) was a component of scoring. 

4. Amount of progress - This response category was coded when the degree of change 
over time in the performance of a task was a component of scoring. 

E4 What environmental elements were used in scoring? 

This multiple-choice item asked about the state’s scoring criteria at the system level, that is, 
the environmental elements that were included in the determination of student scores on the 
alternate assessment. Multiple responses were possible. 

1 . Instruction in multiple settings - This response category was coded when the state 

reported that the extent of instruction conducted in multiple settings was a component of 
scoring. 
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2. Opportunities to plan, monitor, and evaluate work- This response category was coded 
when the state reported students’ engagement in planning, record keeping on their work 
or progress, and evaluating their own performance were components of scoring. 

3. Work with nondisabled peers - This response category was coded when the state 
reported that the degree to which the student was placed in settings with nondisabled 
peers was a component of scoring. 

4. Appropriate human and technological supports - This response category was coded 
when the state reported that the types of aides or assistive technology used during the 
assessment were a component of scoring. 

5. None of the above. 

E5 What types of training were provided for assessment administrators? 

This item asked about the types of training provided to individuals on administering the alternate 
assessment. Multiple responses were possible. 

1 . Non-face-to-face training - This response category was coded when administrators 
were given an administration manual that they used in independent training and/or were 
given administration training support such as videos, Power Point presentations, or 
written guidance online. 

2. Face-to-face training/events/tutorials - This response category was coded when in- 
person was offered by the district or the state on the administration of the alternate 
assessment. 

3. Training was mandatory and/or certification was required - This response category 
was coded when administrators of assessments were required to pass a test and/or 
participate in a tutorial in order to be certified to administer the alternate assessment. 

E6 What types of training were provided for assessment scorers? 

This item asked about the types of training provided to individuals on scoring the alternate 
assessment. Multiple responses were possible. 

1 . Non-face-to-face training - This response category was coded when scorers were given 
a scoring manual that they used in independent training and/or were given scoring 
training support such as videos, Power Point presentations, or written guidance online. 

2. Face-to-face training - This response category was coded when in-person training was 
offered by the district or the state on the scoring of the alternate assessment. 

3. Training was mandatory and/or certification was required - This response category 
was coded when scoring training was mandatory and scorers were required to pass a 
scoring test or verify that they had received training or participated in a tutorial in order 
to be certified to score the alternate assessment. 
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E7 Who received individual student reports? 5 

This multiple-choice item asked about whether individual student reports or other reports were 
provided to parents and/or schools and teachers. 

1 . Parents - This response category was coded when individual student reports were 
provided to parents. 

2. Schools and teachers - This response category was coded when the state provided 
schools and teachers any reports other than what was publicly reported. These 
additional reports may include greater detail in student-level performance data than that 
in public reporting. They also may provide data at the benchmark/indicator levels or 
group students in units helpful for school-level data summary. 

E8 How were individual student results on the alternate assessment expressed? 

This item summarized, at the aggregate level, the results included in individual students’ reports. 
This was a multiple-choice item, and multiple responses were possible. 

1 . State’s achievement standards 

2. Scores (including raw scores, scale scores) 

3. Percentiles 

E9 For whom was interpretive guidance on the alternate assessment developed? 

This item asked about whether interpretive guidance was created for schools, teachers, parents, 
and/or students to provide for a clear understanding and analysis of student performance. This 
was a multiple-choice item, and multiple responses were possible. 

1. School-level administrators 

2. Teachers 

3. Parents 

4. Students 

E10 Information included in reports given to parents 

This item asked about the types of information provided to parents about the alternate 
assessment. Information ranged from student performance level to explanations of descriptors 
and test items. 

1 . Performance/achievement level. 

2. Scores (including raw scores, scale scores, percentiles). 

3. Standard/strand breakouts included information that was more specific than content 
area performance, such as the subdomain level of each content area. 

4. Indicator/benchmark breakouts included information that was more specific than 
standard/strand performance, such as the level of performance indicators or individual 
items. 

5. Performance/achievement level descriptors included descriptors that indicated what it 
means to perform at a particular performance/achievement level. 

6. Sample test items. 



5 Variables E7, E9, and E10 were derived from a series of open-ended items that asked about the criteria for 
reporting and interpreting. These items were combined and organized into topic areas and specific response 
categories for each topic area during coding. The topic areas included information on who received student reports, 
results, and interpretive guidance. 
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